Mar 062011
 

Introduction

At the very beginning I should start with some background information: I am working in a small software company in a team with 13 developers. We continuously developing a software for the medical domain with Qt as UI framework and Visual C++. Our platform is MS Windows. Until autumn 2009 we were developing with the V-model.

As the requirement and priorities of our customer changed many times, we decided to change from the V-model to SCRUM. At the same time we changed from developing all features on the trunk of our SVN repository to feature branching.

As it is recommended in the literature we updated all the feature branches every day and merged the features back, after a story was done. And here the pain stared. SVN is a great SCM repository software, but it is not made for regular merging. So we started to look for alternatives. We put an eye on Git, Hg, Bitkeeper and Perforce. With four colleagues we made a list of requirements, use cases and started an evaluation phase for several weeks. At the end  Git and Hg were the most preferred ones. During this time I looked in the net and searched for pros and contras for both of them. From all the blogs, articles and our own experience i came to the conclusion that Git and Hg offer nearly the same functionality. At the end we decided to go for Hg because of the better support with TortoiseHg, the Windows Explorer integration and the less confusion interface.

In the next chapter I try to describe our way to convert the SVN repository.

SVN Preparation

Our source repository had at that time about 90.000 revisions and its size was about 7GB and the source code had about 500,000 lines of code.

It is recommended in the Mercurial book that one should start the migration from a repository clone. I strongly can recommend to follow that way. As I am more used to Windows I started to to create the mirror within Windows XP. But Subversion does not work stable under Windows XP. The socket layer crashed several time during the cloning process and I had to reboot the system. So I tried to perform the process on a Windows Server 2003 installation. The creation of the mirror was now going well, but later the needed SVN-Dump did not went through. So I installed an Ubuntu in a virtual machine and re-did the complete process.

So if not available in the Linux installation, install svn with

Now create an empty repository

Now initialize the mirror. Be aware if one does not restrict the path to a certain repository path, one gets the complete repository. The repository had about a size of 60GB. So it would need time and space to set this up.

The first time one is asked to accept the certificate and one has of course to enter ones password. Use the correct apostrophe! So not a ‘, but a `!

Now we initiate the sync.

This command can be repeated as often as one like. It always checks then the source repository for updates and appends them. The history and its version numbers are preserved. It is not known if copy from or copy to from outside of the given path are used in the repository.

Migration from Subversion to Mercurial

During the last years we had stored many binary files in the repository which caused the repository to grow to a size of about 7GB. So I decided that we transfer the complete file history of the last three major releases  into Hg and filter with svndump all not needed binaries from the repository. So we wanted to be able to build all versions since the last major release.

The following assumptions are made:

  • The Subversion mirror repository is here
  • The location for the new Mercurial repository is

Patching Hg conversion module

The Subversion conversion tool must be patched because of deficiencies inside the code. Be aware as soon one updates the Subversion packages one has to re-apply the changes! The files is under

Line 353
This line must contain all branches that shall be converted. All others are excluded.
Lines 354-363
These lines must be indended by 3 spaces.
Lines 366-368
These lines must be commented, otherwise the conversion will abort.

Execute the actual conversion command in a shell:

Here is the description of the command in detail:

svn2hg.txt
This file contains all paths which shall be excluded during the conversion, because they contains either binary files or branches, etc. that are not needed any more.
branchmap.txt
This file helps to rename branches to different, better names.
splicemap.txt
This file contains a description of additional parent and child connections. (Currently not working correctly, or I was not able to get it work)
{start revision}
This is the first revision that is taken from the Subversion repository.
user.txt
A file that contains a mapping from the Subversion user names to real names that are used in Hg.

Finalizing

Now it is strongly recommended to do a binary comparison of the head of the Mercurial repository against the latest revision of the Subversion repository to ensure that no conversion errors happened. Be aware that the DVCS do not support empty directories and the DVCS does not support file property makros as Subversion.

Experience

We did a training over several sessions for all users about the differences between SVN and Hg before we used it in production. Then we made a cut from using SVN and started using Hg between two SCRUM sprints and no major issues came up after that. Only once we had the problem that one user used the merging functionality as it is in SVN. So he just wanted to pick a single change set and transfer it to a different branch. So at the end it was a training issue. Today everybody is really satisfied with Hg.

In the next article I will described why we use Hudson / Jenkins as build system and how we get there.

At the end I want to apologize, English is not my first language and this is my very first blog. So any kind of critics or advices to improve is welcome.

  41 Responses to “Why we changed to Mercurial Hg and how we did it”

  1. Hello, Thanks for a great post and the interesting comments.

  2. You saved me a lot of hsalse just now.

  3. Thanks for every other informative site. The place else may just I am getting that type of info written in such an ideal means? I’ve a undertaking that I’m simply now running on, and I’ve been at the glance out for such info.

  4. Hmm it appears like your site ate my first comment (it was super long) so I guess I’ll just sum it up what I wrote and say, I’m thoroughly enjoying your blog. I too am an aspiring blog blogger but I’m still new to everything. Do you have any suggestions for novice blog writers? I’d certainly appreciate it.

    • Felix Petriconi

      Sorry to hear that your blog entry was eaten! I can assure you that I feed my machine enough 😉
      Hmm, I try to write in a way as I like other blog entries or what I would like to read in others blogs.
      A great series of books is published by Pragmatic Programmers and they are just in the process of publishing one regarding tech blogs: http://pragprog.com/book/actb/technical-blogging. Perhaps this helps.
      Take care!

    • Well I’ve set up an account on Github, and will be snuhipg GLESGAE in tonight.I think the best idea for me would be to use Git and Mercurial to do something useful, as that’ll be a better test run than running through any tutorial that artificially creates branches and repositories and merges them about. Granted that’s useful, but it’s in a controlled environment kind of thing.Probably a good thing I have two pressing tasks to get on with as tests, then!1) Sort out some way for the Contexts to be more generic, and the Vertex Buffers to have an option of being plain vertex arrays, or have support for VBOs or even VAOs ( which I’ve found out that iOS prefers through recent work endeavours. )2) I also need Win32 support and Audio of some description so they can be lumped together as they’re likely as much a unit of work as sorting out the rendering bits.At any rate, knowing how to use Git, Mercurial and SVN is probably a good thing to have anyway, so the month of fiddling won’t be wasted regardless of what I end up using

      • Thanks for the feedback, Jakub.If you peefrr Trac, you can always contribute to Trac’s GitPlugin (and GitwebPlugin).If you don’t like gitweb, you can always improve it, or use one of the alternate git web interfaces (cgit, wit, git-php). Don’t get me wrong it’s not that I don’t like gitweb, it’s just that I peefrr Trac (after all, it’s much more than just a read-only view of a repository) and to a lesser extent SVN::Web.Trac obviously has much more features than gitweb due to its different goals and objectives. Regarding code contribution, I honestly lack the time to do that (and I suspect I also lack the knowledge of how Trac Co. are working behind the scenes). But I can give verbal feedback if that counts BTW. what features does SVN::Web has that gitweb lacks; what can be improved in gitweb?One of the things that SVN::Web has over gitweb is that you can easily get an RSS feed with news/information of the current view. I have yet to find a way to get just an RSS feed for a single file or for a directory (the latter being recursive down the directory tree).Example:* * The available RSS feed in the first link automatically filters only relevant information for deliciousapi.py whereas gitweb’s RSS feed is the global one you get everything unfiltered. It’s the same for this blog: you have a and an .The next thing is that SVN::Web’s URLs are much nicer to read. Yes, that’s maybe only a cosmetic issue, but I really peefrr, say, clean URLs. (maybe there is an easy way to get the same setup for gitweb but I haven’t found it yet)SVN::Web also allows to , not just .On a related note, the nomenclature (?) of SVN::Web seems much more familiar to general users (I know, a fuzzy definition). At other projects, i.e. not this website, where I use gitweb I get asked by users where is the link to browse the directories and files? (= tree) or how do I see the changes of a commit? (=commitdiff; they seem to insist on clicking just commit and getting stuck there). For me, gitweb’s nomenclature is fine I know the basics of git and its concept/design, so I easily find my way around. I have found this not to be true for users who just want to use gitweb as an interface to a repository but who are not really familiar with git. I remember some related discussions on the git mailing list which I follow occasionally.To sum it up, I rate SVN::Web’s usability a bit higher than gitweb’s at the moment. I mean, I sometimes get emails from users who download, say, my del.icio.us Python API and then they ask me how to iterate over a list in Python But since I also want to help the beginners, I try to find suiting solutions including a repository web UI. In the case of the latter though, I still decided to switch to git/gitweb because continuing using Subversion just for the nicer SVN::Web interface was too tedious in my daily routine.I hope this gives some insight into my previous blog post which I hope did not sound like bashing gitweb.Best wishes, Michael

  5. I’m impressed, I have to say. Truly hardly ever do I come across a blog site that’s both educative and entertaining, and let me tell you, that you have hit the nail on the head. Your concept is exceptional; the issue is something that not enough folks are speaking smartly about. I am extremely happy that I came across this blog

    • I use Git both at work and for personal poerjcts. Its superior branch merging and local branching capability are great advantages over Subversion. Git is very powerful but as a result has lots and lots of commands, options and features that are confusing and unnecessary for everyday use. However I found that once you learn how to do the basics, you can get by perfectly fine. My advice would be to seek out a friendly tutorial and avoid the official documentation as a learning resource it’s just confusing!

      • I’m sure a few years ago people said “SVN isn’t beettr than CVS, it’s just different”.You must have never used CVS, no one ever made a claim like this.Implicit your post is that not being able to modify tags is a good thing, which I suspect probably falls inline with what many people would expect. I wonder if the devs had a particular use case in mind when that decision was made.

  6. Many thanks for an incredible post, can see your others topics. i appreciate your notions on this, I experienced a bit thump by this post. Merit again! You make a great moment. Has excellent data here. I feel if more people consideration for it like that, they’d have got a better time have the suspend ofing the matter.

    • im having prmeobls running avida, i have downloaded 2.12.1 for my mac. i have also set up github to my computer. when i try to run avida i geterror: unable to open file avida.cfg’.error: configuration file avida.cfg’ not foundlogout[Process completed]im not sure were im going wrong, if someone could help or send me in the right direction it would be greatly appreciated.Thanks in advance.

  7. Aw, it was a truly great article. In idea I would really like to put in writing like this additionally – taking time and actual energy to create a very good article… but what can I say… I procrastinate alot and by no means seem to get something done.

  8. It is really a great and useful piece of info. I’m glad that you shared this useful info with us. Please keep us informed like this. Thank you for sharing.

  9. Hiya. Very nice blog!! Guy .. Beautiful .. Wonderful .. I will bookmark your site and take the feeds also…I am happy to find a lot of useful info right here within the article. Thanks for sharing.

  10. It¡¦s actually a cool and helpful piece of information. I am satisfied that you shared this useful info with us. Please keep us informed like this. Thank you for sharing.

  11. I think other site owners should take this site as an model – very clean and great style and design, as well as the content. You are an expert in this topic!

  12. This really answered my problem, thank you!

  13. quite valuable material, all in all I imagine this is worthy of a bookmark, thanks a lot

    • Well i’m not sure if there are any better way for do this, but i think if you don’t want to dwnlooad then you need to upload.Because the key is of that problem is about the synchronization between working copy and the repository.So the other way around is after you export it to the clean one, you need to delete your working copy and repository folder and add your clean folder which you exported to the subversion repository.But I never do that before, because i think the latest updated one is at my remote subversion repository. So, if you sure that the latest one is on your computer then you can try this way.

      • The WordPress svn repo also has tags for each release. You can swtcih from one tag to another, so I’m not sure I understand your reasoning here. 🙂

  14. nice site the text becomes a bit hard to read on your background after a while tho. jus a friendly suggestion =)

  15. I do consider all the concepts you have presented to your post.

    They are really convincing and can certainly work. Still,
    the posts are too quick for beginners. Could you please extend them a bit from subsequent time?
    Thank you for the post.

  16. I do not even know how I ended up here, but
    I thought this post was good. I do not know who you are but definitely you’re going to a famous blogger if you aren’t already 😉 Cheers!

  17. Greetings! Very helpful advice within this post!
    It’s the little changes that produce the most important changes. Thanks for sharing!

  18. When someone writes an paragraph he/she keeps the idea of a user in his/her brain
    that how a user can know it. Therefore that’s why this post is perfect. Thanks!

  19. Loving the informаtion on thiѕ website, you’ve done outstanding job concerning the posts.

  20. Rеgards with thiѕ pοѕt, I am а big fаn
    οf these site wοuld lіke to сontinuе updateԁ.

  21. I’ve been browsing online more than 3 hours today, yet I never found any interesting article like yours. It’s pretty worth enough for me. In my view, if all site owners and bloggers made good content as you did, the internet will be a lot more useful than ever before.

    • Hi Scott,For my personal one man preojcts I work on from home I like to keep it simple, I use a hosted subversion solution, check out sourcerepo dot com, costs me 3.95 per month, no config required, it was all up and running in about 2 minutesI use DW at home so very little config was needed as it has its own simple subversion client.for a one man team its fine and does all I needif you’re an eclipse person ya can use subeclipse etc or tortoise or whatever really.if you want to learn more about SVN then install it yourself, but be sure to backup you repo etc.hope this helps

  22. Hi my family member! I want to say that this article is amazing, nice written and come with approximately all important infos. I would like to peer extra posts like this .

  23. Hi, i think that i saw you visited my web site so i came to “return the favor”.I’m trying to find things to improve my web site!I suppose its ok to use a few of your ideas!!

  24. Magnificent goods from you, man. I’ve understand your stuff previous to and you are just too excellent. I really like what you’ve acquired here, certainly like what you are stating and the way in which you say it. You make it entertaining and you still take care of to keep it sensible. I can’t wait to read much more from you. This is actually a terrific site.

  25. I don’t even know how I ended up here, but I thought this post was good. I don’t know who you are but definitely you are going to a famous blogger if you aren’t already 😉 Cheers!

  26. Hey there, You have done a fantastic job. I will certainly digg it and personally suggest to my friends. I’m sure they’ll be benefited from this site.

  27. Hey there and thank you for your info – I have definitely picked up anything new from right here. I did however expertise several technical points using this web site, as I experienced to reload the website lots of times previous to I could get it to load properly. I had been wondering if your web hosting is OK? Not that I am complaining, but slow loading instances times will often affect your placement in google and could damage your quality score if ads and marketing with Adwords. Anyway I am adding this RSS to my e-mail and could look out for a lot more of your respective exciting content. Make sure you update this again very soon..

  28. Thank you for taking the time to write this post. It was most informative and has clearified a feww things for me.

  29. I’ve been exploring for a bit for any high-quality articles or blog posts on this kind of space . Exploring in Yahoo I eventually stumbled upon this site. Studying this information So i’m
    satisfied to exhibit that I have a very good uncanny feeling I came upon exactly what I needed.
    I such a lot unquestionably will make certain to
    don?t overlook this web site and give it a glance regularly.

    my weblog … bik czyszczenie

  30. Simply desire to say your article is as amazing. The clarity in your post is simply great and i could assume you are an expert on this subject. Well with your permission let me to grab your feed to keep up to date with forthcoming post. Thanks a million and please carry on the gratifying work.

  31. Thank you for the auspicious writeup. It in fact was a amusement account it. Look advanced to more added agreeable from you! However, how can we communicate?

 Leave a Reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">

(required)

(required)