This page recap all the open questions for the migration to the git tool for managing source code and their answers.
We all agree on the fact that a giant, important step like this requires coordination, and the feedback/contribution of many people as possible, especially current maintainers. The process of updating the source code management system will be split into different sequential phases. Each phase will be used as input for the next stage.
Phase 1 (done): Definition of questions to be answered: Each ns-3 users can put questions on this page that he/she would like to have it answered. The problems can have multiple points. Users can add points to each item.
Phase 2 (done): Live talks in wns3: In this week the proposal can be discussed live.
Phase 3 (done): Discussion of options: Each developer starts answering the questions (even not his/her own) with his/her proposal. Each proposal can be refined, discussed on the mailing list, and so on. Plan to do this at Google Hangout on October 10, 15:00 UTC (contact email@example.com if interested).
Phase 4 (done): list discussion: If there are questions with conflicting proposals, the current maintainers will have a vote on what plan should be implemented.
Phase 5 - Implementation: The maintainers will start migrate the source code. At the end of the phase, the community will give comments. Discuss, integrate, iterate until final release.
Each ns-3 user was free to add questions that he/she would like to have answered. The objective was to have a coherent set of stakeholder interests, as well as a set of design rules that take into considerations these interests. The reality is that we have a different way of working, and it is difficult to have something that is good enough for everyone. Below were the questions raised and conclusions later drawn.
- (nat) Should we move some modules to move to contrib? <- No; this can be done as a subsequent step
- (nat) What modules should we move to contrib? <- None
- (nat) Define how the contrib/ modules will be fetched and compiled <- Using bake or plain download
- (nat) Define the service we will use (GitHub, GitLab, BitBucket) <- GitLab
- (nat) Define the official branches (stable releases, development) <- See workflow discussion below
- (nat) Define how the versions are managed and released <- See workflow discussion below
- (nat) Define a workflow for integrating patches: <- See workflow discussion below
- (nat) stable fixes (fixes that are for an already shipped version)
- (nat) development fixes/features (fixes and features for master branch)
- (nat) Define the accepted code review models <- GitLab Merge Review, others (GitHub, code review) can still be used
- (nat) Define the permissions of each user inside the projects (should everyone have r/w access to the repo?) <- Use GitLab groups to manage access
- (rediet) Define how issues/bugs will be reported (e.g. special rights needed?) <- Will migrate to GitLab tracker
- (rediet) Will current features (currently managed through wiki) be handled by service (e.g. artefact)? <- Keep wiki and static server, start to use GitLab features (CI/CD, GitLab pages, tracker) over time.
Phases 2, 3, 4
Historic details omitted herein but after discussion throughout the summer, including list discussion and offlist maintainer discussions, and testing of three popular cloud-hosted services (GitHub, GitLab, Bitbucket), decision was taken to use GitLab cloud (GitLab.com) as hosting service.
We plan to migrate to GitLab.com by year's end. Below is a proposed strawman of things to be done. Natale has volunteered to implement the main things.
The main tree will be at https://gitlab.com/nsnam/ns-3-dev.git
Bake will have a tree at https://gitlab.com/nsnam/bake.git
Static web site (jekyll) will be at https://gitlab.com/nsnam/nsnam-www.git
Open issue: should we continue to use ns-3-allinone or transition to bake? If we continue to use ns-3-allinone, suggest to put a repository at https://gitlab.com/nsnam/ns-3-allinone.git
- Suggested resolution to issue: Migrate also the 'ns-3-allinone' repository; it may continue to be used for holding scripts like dist.py for building source release tarballs, regardless of whether we continue with build.py.
No subgroups initially; subgroups may be created in the future for documentation not already maintained with ns-3-dev (e.g. tutorial translations, install guides), or for ns-3 modules for the app store.
Branch management and workflow
Peter has suggested to consider the gitflow workflow (https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow). This would seem to align with our release practices (so long as we introduce also the 'support' branches described in this StackOverflow issue: https://stackoverflow.com/questions/16386323/following-git-flow-how-should-you-handle-a-hotfix-of-an-earlier-release
Currently a simplified workflow, roughly resembling our Mercurial workflow, is adopted:
- Commit obvious non-critical fixes (documentation improvements, typos etc.) directly into the master branch.
- For each new major feature, create a feature branch in your own fork
- Merge feature branches into master after review.
- Tag releases wherever they are, either on the master or support/release branch. Tags are visible globally in git, they are not commited to special .hgtags like in Mercurial.
- If minor releases are required, create support branch for the major release and push fixes there. Tag minor releases accordingly.
- If hotfix needs a review, start a "hotfix" branch and create MR.
- Merge support branch back into master if hotfix is relevant.
The primary review mechanism will be for contributors to create a GitLab Merge Request by forking ns-3-dev and using GitLab tools to generate and update merge requests. At the end of this process, maintainers will work with the submitter to create either a squash merge or full merge as appropriate.
Small patches (a few lines) could still be handled with raw diff/patch files in the tracker.
Open issue: Do we allow pull requests on GitHub and Bitbucket, and use tools there (if the maintainer agrees to perform reviews there) or do we request that they all come to GitHub merge requests? To date, we have not encouraged these, but many GitHub users may find it easier to submit. Perhaps there is an easy workflow for maintainers to rebuild GitHub pull requests on GitLab as merge requests.
- Suggested resolution to issue: Do not have duplicate issues on GitHub and GitLab. Try to migrate issues to GitLab. GitHub and Bitbucket can stay as mirror. Consider to expire PRs after some time (3 months?), after which the submitter must resubmit.
Open issue: Do we allow people to continue to use codereview.appspot.com for some large reviews? We have some active reviews there and it may be useful to have as a backup service also.
- Suggested resolution to issue: Allow codereview.appspot.com reviews for the time being, until if and when we are comfortable that the Merge Review process can replace it. Do not duplicate (mirror) Merge Review and codereview issues, to avoid confusion about where to review.
Need volunteer to experiment with daily builds on GitLab.com CI infrastructure, and to propose a workflow allowing for CI testing before final commit. This step is not needed to make the cutover (can still rely on existing Jenkins).
New issues will go to the ns-3-dev tracker on GitLab.com. This needs to be configured with email hook to firstname.lastname@example.org.
Previous issues will stay on ns-3 Bugzilla for now; may export and import to GitLab at a future date.
Tutorial must be updated (Getting Started section).
New documentation for end-user best practices for using GitLab.com with ns-3 should be written.
- How to fork and generate a merge request (walk through the lifecycle including iterations on a code review)
- Recommended branch management (when to (not to) squash, rebase, create a new branch, delete a branch, etc.)
- How to submit an issue to the issue tracker
New documentation for maintainers best practices using git and GitLab.com should be written.
- When to squash merge or not
- How to test a proposed Merge Request
- How to handle code that comes from other sources (GitHub, bugzilla, codereview)
- what not to do (e.g. rebase on public branches)
This Mercurial service will remain open, but pushes to bake and ns-3-dev will be disabled.