Difference between revisions of "HOWTO use Mercurial Queues to manage your ns-3 patches"

From Nsnam
Jump to: navigation, search
(HOWTO develop on multiple machines)
 
(10 intermediate revisions by the same user not shown)
Line 50: Line 50:
 
   hg pull http://code.nsnam.org/ns-3-dev
 
   hg pull http://code.nsnam.org/ns-3-dev
 
   hg update
 
   hg update
   hg qpush -m project.patch
+
   hg qpush project.patch
Note that you really want to merge your patch back in, so make sure you provide the '''-m''' option to "'''hg qpush'''" which will trigger a three-way merge if the patch fails to apply.
+
Note that you are going to try and merge your patch back in, so you might get rejects that must be resolved.
  
 
10. You can go back and repeat steps 6-9 until you think you are ready for review.
 
10. You can go back and repeat steps 6-9 until you think you are ready for review.
Line 83: Line 83:
 
   hg qinit -c
 
   hg qinit -c
 
   hg qnew project.patch
 
   hg qnew project.patch
2. Now pop the patch you just created so that it is not applied in the repository
+
2. Now pop the patch you just created so that it is not applied in the repository (there is nothing in the patch -- we are just changing the status)
 
   hg qpop -a
 
   hg qpop -a
 
3. Now change into the .hg/patches directory of your new working repository and pull down your saved patches
 
3. Now change into the .hg/patches directory of your new working repository and pull down your saved patches
 
   cd .hg/patches
 
   cd .hg/patches
 
   hg pull http://code.nsnam.org/your-user-name/ns-3-project-patches
 
   hg pull http://code.nsnam.org/your-user-name/ns-3-project-patches
 +
  hg update
 
4. Now apply your saved patches
 
4. Now apply your saved patches
 
   hg qpush project.patch
 
   hg qpush project.patch
 +
Note: If the ns-3-dev repository has changed since you last pushed your patch, you may get rejects which you will have to resolve manually.
 +
 
5. Go back and do steps 6-9 of [[#HOWTO use Mercurial Queues to manage a patch from creation to final push]] above until you are done with the second machine.  Make sure your patch is qcommittted and backed up as in step 8 above before going home for the night.
 
5. Go back and do steps 6-9 of [[#HOWTO use Mercurial Queues to manage a patch from creation to final push]] above until you are done with the second machine.  Make sure your patch is qcommittted and backed up as in step 8 above before going home for the night.
  
Line 100: Line 103:
 
   hg qinit -c
 
   hg qinit -c
 
   hg qnew project.patch
 
   hg qnew project.patch
2. Now pop the patch you just created so that it is not applied in the repository
+
2. Now pop the patch you just created so that it is not applied in the repository (there is nothing in the patch -- we are just changing the status)
 
   hg qpop -a
 
   hg qpop -a
 
3. Now change into the .hg/patches directory of your new working repository and pull down your saved patches
 
3. Now change into the .hg/patches directory of your new working repository and pull down your saved patches
 
   cd .hg/patches
 
   cd .hg/patches
 
   hg pull http://code.nsnam.org/your-user-name/ns-3-project-patches
 
   hg pull http://code.nsnam.org/your-user-name/ns-3-project-patches
4. Now apply your recovered patches
+
  hg update
 +
4. Now apply your recovered patches.
 
   hg qpush project.patch
 
   hg qpush project.patch
 +
Note: If the ns-3-dev repository has changed since you last pushed your patch, you may get rejects which you will have to resolve manually.
 +
 
5. Breathe again and go back to steps 6-9 of [[#HOWTO use Mercurial Queues to manage a patch from creation to final push]] above -- you are saved.
 
5. Breathe again and go back to steps 6-9 of [[#HOWTO use Mercurial Queues to manage a patch from creation to final push]] above -- you are saved.
  
 
----
 
----
  
[[User:Craigdo|Craigdo]] 21:20, 13 May 2009 (UTC)
+
[[User:Craigdo|Craigdo]] 05:15, 24 June 2009 (UTC)

Latest revision as of 20:01, 21 August 2009

Main Page - Current Development - Developer FAQ - Tools - Related Projects - Project Ideas - Summer Projects

Installation - Troubleshooting - User FAQ - HOWTOs - Samples - Models - Education - Contributed Code - Papers

One of the most annoying aspects of developing in an an open-source environment such as ours is managing patches. For small patches it is really not a big issue, but if you are developing a patch over a considerable amount of time and work from multiple hosts with an unclear backup policy you really need a way to deal with the underlying issues.

The simplest way to do this is to use Mercurial as a "fancy version enabled backup system" as another one of our developers put it. This has the serious disadvantage of putting your development-related commits in the history of your change and then ultimately into the history of the project. This is annoying noise to any reviewer who is only interested in the final result, not that you fixed a bug months before the review. It is unpleasant noise to a future developer who is trying to bisect the history to find when a bug was introduced.

The following process allows you to use Mercurial as a backup system, allows you to version your commits during development. It also allows you to easily post patches for reveiw and produce a final result without all of that commit noise you generated during development.

I make no guarantees about this process -- use at your own risk and be very, very careful about losing work until you really know what you are doing. Try it all out before you do anything "real" as you can lose work if you mess up. Take a look in the resulting .hg/patches directory to understand what is happening as you progress. There are many more features of Mercurial Queues available, but this process lets me work the way I prefer, and does not fill up histories with junk and makes clean patches possible.

YMMV.

HOWTO use Mercurial Queues to manage a patch from creation to final push

1. Before you start, you need to edit your ~/.hgrc file and enable the Mercurial Queues extensions

 [extensions]
 hgext.mq =

2. Create a repository on your code server that you want to use as the backup location. I typically prefix my directory name with "ns-3-" followed by some project name and then "-patches" to indicate that this is a Mercurial Queues patches repository. So given that I'm going to be working on a project called "project" I would personally create a repository on code.nsnam.org called ns-3-project-patches

 ssh code.nsnam.org
 cd repositories/your-user-name
 mkdir ns-3-project-patches
 cd !$
 hg init
 exit

3. Back on your working machine, clone the repository you want to use as a basis for your patch. Typically this will be ns-3-dev. My method of operation is to use the same naming convention in my working repository as I did on the code server (in reality I would use ns-3-allinone to do this)

 hg clone http://code.nsnam.org/ns-3-dev ns-3-project

4. Next you have to enable the repository to use Mercurial Queues and also create the nested repository which will be your backup repository.

 cd ns-3-project
 hg qinit -c

5. Tell Mercurial Queues that you are starting a new patch (remember to substitute the real name for your project). The message here will eventually turn into your commit message for the final patch. I'll show you how to change this message later, BTW.

 hg qnew -m "message describing purpose for patch" project.patch

6. Begin working on your patch.

7. Periodically, you can checkpoint your work and save your progress into the patch you are creating. This doesn't do any kind of saving or pushing, it just updates the patch you are in the process of creating.

 hg qrefresh

8. Once you have enough done that you want to back it up, then you package it all up and push it out to your backup location (remember to substitute your user and project name).

 hg qcommit -m "checkpoint message"
 cd .hg/patches
 hg push ssh://code.nsnam.org//home/your-user-name/repositories/your-user-name/ns-3-project-patches

It is this step that actually creates the fancy version enabled backup. Check out #HOWTO recover from disaster for disaster recovery instructions.

9. If the project lasts long enough, you may end up out-of-date with respect to the base repository (cf. ns-3-dev). In this case you will need to rebase. You do this by removing your patch from the working repository, pulling in the possibly changed bits and then reapplying your patch. Substitute the appropriate base repo and project name.

 hg qpop -a
 hg pull http://code.nsnam.org/ns-3-dev
 hg update
 hg qpush project.patch

Note that you are going to try and merge your patch back in, so you might get rejects that must be resolved.

10. You can go back and repeat steps 6-9 until you think you are ready for review.

11. When you are ready to generate a patch for review, make sure you are rebased and simply

 hg qdiff > patchfilename

12. Send the patch out by email or put it up using a review tool.

13. After the flames die down and you have reformatted all of your code to agree with the ns-3 coding standard, you will probably have done several more iterations of steps 6-9. You can now give your final patch a commit message (if you need a different one than the one you put in to start back in step 4).

 hg qrefresh -e

This will open up the "commit message" in your favorite editor and allow you to change it. If you didn't bother to add one in step 5 you can add one here.


WARNING WARNING WARNING: Be very careful in the next step. If you use "hg qdelete" by itself you will throw away your work. What you want to do is "hg qdelete -r" (note the -r option) which makes your patch permanent and then discards the patch. That's too insanely on the edge of disaster for me, so I would probably alias the "qdelete -r" command to something like "qfinish."


14. Now you can turn your patch into a permanent changeset in your working repository.

 hg qdelete -r project.patch

15. Finally, you can push your changes into ns-3-dev (assuming you have the proper credientials).

 hg push ssh://code@code.nsnam.org//home/code/repos/ns-3-dev

16. Go to Disneyland.

HOWTO develop on multiple machines

I sometimes need to move from machine to machine during my patch development. This is useful when you discover problems on another machine/OS type (you do make sure your patches run on all of our hosts before pushing them to ns-3-dev, right). This can be easily done by pushing your patches and then pulling them into another machine.

1. Go to your second machine and clone a copy of the repository you started from and repeat the mercurial queues initialization you did to start the process (use ns-3-allinone if you like, instead of cloning the repo)

 hg clone http://code.nsnam.org/ns-3-dev
 hg qinit -c
 hg qnew project.patch

2. Now pop the patch you just created so that it is not applied in the repository (there is nothing in the patch -- we are just changing the status)

 hg qpop -a

3. Now change into the .hg/patches directory of your new working repository and pull down your saved patches

 cd .hg/patches
 hg pull http://code.nsnam.org/your-user-name/ns-3-project-patches
 hg update

4. Now apply your saved patches

 hg qpush project.patch

Note: If the ns-3-dev repository has changed since you last pushed your patch, you may get rejects which you will have to resolve manually.

5. Go back and do steps 6-9 of #HOWTO use Mercurial Queues to manage a patch from creation to final push above until you are done with the second machine. Make sure your patch is qcommittted and backed up as in step 8 above before going home for the night.

HOWTO recover from disaster

If you have backed up your patches and you make a horrible mistake or you computer crashes, you can easily recover. All you need to do is to get a copy of your base repository and then re-apply your saved patch (this is really exactly the same process that you use to move between machines, of course, since that is ultimately what you are doing if your machine dies).

1. First, get your computer fixed and then clone a copy of the repository you started from and repeat the mercurial queues initialization you did to start the process.

 hg clone http://code.nsnam.org/ns-3-dev
 hg qinit -c
 hg qnew project.patch

2. Now pop the patch you just created so that it is not applied in the repository (there is nothing in the patch -- we are just changing the status)

 hg qpop -a

3. Now change into the .hg/patches directory of your new working repository and pull down your saved patches

 cd .hg/patches
 hg pull http://code.nsnam.org/your-user-name/ns-3-project-patches
 hg update

4. Now apply your recovered patches.

 hg qpush project.patch

Note: If the ns-3-dev repository has changed since you last pushed your patch, you may get rejects which you will have to resolve manually.

5. Breathe again and go back to steps 6-9 of #HOWTO use Mercurial Queues to manage a patch from creation to final push above -- you are saved.


Craigdo 05:15, 24 June 2009 (UTC)