|
|
| Line 15: |
Line 15: |
| | 2. '''(development tool)''' Help developers better manage multiple builds. | | 2. '''(development tool)''' Help developers better manage multiple builds. |
| | | | |
| − | = ns-3.17 goals = | + | = ns-3.22 goals = |
| | | | |
| − | Below are proposed goals for ns-3.17. | + | Below are proposed goals for ns-3.22. |
| | | | |
| − | '''1. bake.py provides existing download.py and build.py with no loss of functionality''' | + | '''1. initial support for third-party ns-3 modules''' |
| | | | |
| − | '''1.1''' For ns-3.17 tarball release:
| + | Several (DTN, epidemic routing, routes mobility model, NSC, perhaps others ...) will become available only through the bake tool; large modules will no longer be merged to ns-3-dev but will need to be selected and managed by the bake tool. |
| | | | |
| − | The proposal is to ship, in the allinone tarball, a bakeconf.xml that allows the user to fetch other components (including DCE), and source for ns-3.17, pybindgen, and netanim.
| + | This has implications for how we manage Doxygen, Sphinx documentation, the scratch directory, waf, etc. |
| | | | |
| − | tar xjf ns-allinone-3.17.tar.bz2
| + | '''2. removal of 'download.py', 'build.py', and 'dist.py' in ns-3-allinone; replace with bake commands.''' |
| − | cd ns-allinone-3.17
| + | |
| − | ./bake.py configure -e ns-3.17
| + | |
| − | ./bake.py build
| + | |
| | | | |
| − | '''note:''' we could also ship a prebuilt bake configuration file for ns-3.17, and users could skip the configure step | + | '''3. some existing ns-3-dev components (mesh, wimax, uan) are disabled by default. Provide an improved, unified way for users to control which modules are in/out of the build, on a per-module granularity.''' |
| | | | |
| − | '''1.2''' For ns-3-dev (after merge):
| + | This means reconciling the existing .ns3rc and bake way of controlling the build. |
| | + | = Post ns-3.22 goals = |
| | | | |
| − | hg clone http://code.nsnam.org/bake
| + | It has been previously discussed to break apart ns-3-dev into separate repositories going forward, with a completely flat directory structure. This is for further study beyond ns-3.22. |
| − | cd bake
| + | |
| − | ./bake.py configure -e ns-3-dev
| + | |
| − | ./bake.py download
| + | |
| − | ./bake.py build
| + | |
| | | | |
| − | | + | Trying to integrate the above with binary package distribution (.deb, .rpm, etc.) is out of scope for ns-3.22. |
| − | '''1.3''' existing download.py and build.py
| + | |
| − | | + | |
| − | We keep existing download.py and build.py files, but they print out a backward-compatibility warning and call the equivalent bake commands; e.g.
| + | |
| − | | + | |
| − | cd ns-allinone-3.16
| + | |
| − | ./build.py
| + | |
| − | "Warning: build.py is deprecated, please use './bake.py configure -e ns-3.17 && ./bake.py build' instead.
| + | |
| − | | + | |
| − | '''2. different levels of verbosity show different amount of build information'''
| + | |
| − | | + | |
| − | ./bake.py install
| + | |
| − | | + | |
| − | yields something sparse, such as
| + | |
| − | | + | |
| − | >> Downloading ns-3-dev
| + | |
| − | >> Download ns-3-dev - OK
| + | |
| − | >> Building ns-3-dev
| + | |
| − | >> Built ns-3-dev - OK
| + | |
| − |
| + | |
| − | while this:
| + | |
| − | | + | |
| − | ./bake.py install -vv
| + | |
| − | | + | |
| − | yields the full build output (full waf configure and build information).
| + | |
| − | | + | |
| − | '''3. Managing multiple ns-3 versions and branches'''
| + | |
| − | | + | |
| − | Maintainers commonly work with multiple builds. This can be most easily accomplished using subdirectories.
| + | |
| − | | + | |
| − | hg clone http://code.nsnam.org/daniel/bake
| + | |
| − | | + | |
| − | Put bake main directory on your path and on python path
| + | |
| − | | + | |
| − | export PATH=$PATH:`pwd`/bake
| + | |
| − | export PYTHONPATH=$PYTHONPATH:`pwd`/bake
| + | |
| − | | + | |
| − | '''Note:''' It is an open question whether bake/ is separate from ns-3-allinone/ repository, or part of ns-3-allinone/ repository. Continuing with this example:
| + | |
| − | | + | |
| − | mkdir ns-3-dev
| + | |
| − | cd ns-3-dev
| + | |
| − | bake.py configure -e ns-3-dev
| + | |
| − | bake.py download
| + | |
| − | bake.py build
| + | |
| − | | + | |
| − | and a little later:
| + | |
| − | | + | |
| − | cd ..
| + | |
| − | mkdir ns-3.17
| + | |
| − | cd ns-3.17
| + | |
| − | bake.py configure -e ns-3.17
| + | |
| − | bake.py download
| + | |
| − | bake.py build
| + | |
| − | | + | |
| − | The resulting directory three then contains bake itself, and each repo or branch of ns-3. Within each of these ns-3 subdirectories, there is a source/ and a build/ directory corresponding to that configuration.
| + | |
| − | | + | |
| − | '''4. Managing debug, optimized, and static builds'''
| + | |
| − | | + | |
| − | See the previous item. Users can maintain separate ns-3-dbg, ns-3-opt, ns-3-static, etc. directories. It is just a matter of calling the configuration with a -p dbg, for example. This will make use of the debug predefined configuration and will add -g and create a distinct binary directory for the ns-3 binaries and libraries.
| + | |
| − | | + | |
| − | For example for ns-3.17:
| + | |
| − | bake.py configure -e ns-3.17 -p dbg
| + | |
| − | bake.py build
| + | |
| − | | + | |
| − | The new binary directory build_dbg was created on the same level of the regular build.
| + | |
| − | | + | |
| − | '''5. Passing configure options to bake, especially ns-3 ones such as --disable-python'''
| + | |
| − | | + | |
| − | Each module (nsc, netanim, etc.) has its own set of configuration options. The way that bake presently handles this is by editing the bakeconf.xml file directly (or by setting a predefined module; see below). It may be possible to pass configuration options down to ns-3 such as done with build.py presently. However, this complicates bake, so it is proposed for now that any options be configured via environment variables, if possible, or by creating personal predefined configurations or, in the last case by editing attributes in bakeconf.xml. For example:
| + | |
| − |
| + | |
| − | <attribute name="configure_arguments" value="configure --prefix=$INSTALLDIR --enable-examples
| + | |
| − | --enable-tests --with-nsclick=$SRCDIR/click --with-pybindgen=$SRCDIR/pybindgen --with-
| + | |
| − | openflow=$SRCDIR/openflow-ns3 --with-nsc=$SRCDIR/nsc"/>
| + | |
| − | | + | |
| − | '''6. cleaning the directory'''
| + | |
| − | | + | |
| − | * './bake.py uninstall' will remove the build/ directory and any build artifacts;
| + | |
| − | * './bake.py distclean' will restore the directory to the initial state
| + | |
| − | | + | |
| − | The details need to be worked out further but the intent is to make 'uninstall', 'clean', and 'distclean' work like on other projects.
| + | |
| − | | + | |
| − | '''7. Configuration manipulation'''
| + | |
| − | | + | |
| − | The act of configuring bake will leave a "bakefile.xml" file storing the state of the configuration. This can be further manipulated by enabling/disabling options.
| + | |
| − | | + | |
| − | For example, consider the user who has configured ns-3-dev but wants to deselect/disable netanim. This sequence of commands can accomplish that:
| + | |
| − | | + | |
| − | bake.py configure -e ns-3-dev
| + | |
| − | bake.py fix-config -d net_anim
| + | |
| − | | + | |
| − | Some time later, the following command:
| + | |
| − | | + | |
| − | bake.py configure -e ns-3-dev
| + | |
| − | | + | |
| − | will reconfigure and overwrite entirely the bakefile.xml.
| + | |
| − | | + | |
| − | It is also possible for users to store their own custom configs and modules in a .bakerc file that can be colocated with the current directory or the user's home directory. For example, consider the user who wants a minimal ns-3 with only the wifi module enabled. He or she could populate the following in a .bakerc file:
| + | |
| − | | + | |
| − | <predefined name="ns3-min">
| + | |
| − | <enable name="ns-3-dev"/>
| + | |
| − | <disable name="nsc"/>
| + | |
| − | <disable name="net_anim"/>
| + | |
| − | <disable name="pybindgen"/>
| + | |
| − | <disable name="click"/>
| + | |
| − | <disable name="openflow-ns3"/>
| + | |
| − | <append module="ns-3-dev" name="configure_arguments" value=" --enable-modules=core,network,wifi"/>
| + | |
| − | </predefined>
| + | |
| − | | + | |
| − | '''8. Show behavior'''
| + | |
| − | | + | |
| − | bake.py shows available packages, whether they are enabled, and their dependencies; e.g.:
| + | |
| − | | + | |
| − | ./bake.py show --enabled
| + | |
| − | module: click (enabled)
| + | |
| − | No dependencies!
| + | |
| − | module: openflow-ns3 (enabled)
| + | |
| − | No dependencies!
| + | |
| − | module: gccxml-ns3 (enabled)
| + | |
| − | No dependencies!
| + | |
| − | module: nsc (enabled)
| + | |
| − | No dependencies!
| + | |
| − | module: qt4 (enabled)
| + | |
| − | No dependencies!
| + | |
| − | module: pygccxml (enabled)
| + | |
| − | depends on:
| + | |
| − | gccxml-ns3 (optional:True)
| + | |
| − | module: net_anim (enabled)
| + | |
| − | depends on:
| + | |
| − | qt4 (optional:True)
| + | |
| − | module: pybindgen (enabled)
| + | |
| − | depends on:
| + | |
| − | pygccxml (optional:True)
| + | |
| − | module: ns-3-dev (enabled)
| + | |
| − | depends on:
| + | |
| − | net_anim (optional:True)
| + | |
| − | nsc (optional:True)
| + | |
| − | pybindgen (optional:True)
| + | |
| − | click (optional:True)
| + | |
| − | openflow-ns3 (optional:True)
| + | |
| − | | + | |
| − | In the example above, the gccxml-ns3 dependency is shown for pygccxml. It is out of scope for bake.py to manage package dependencies for some of these (e.g. showing "scons" as a dependency for nsc). However, if the user does not have a required tool, during the course of the build, bake reports this to the user. If the developer explicitly created a system dependency, for example, it is possible to add a message so that the user can try to find/install the missing package.
| + | |
| − | | + | |
| − | The user can check the missing bake dependencies by calling:
| + | |
| − | | + | |
| − | bake.py check
| + | |
| − | > Python - Ok
| + | |
| − | > Mercurial - Ok
| + | |
| − | > CVS - Ok
| + | |
| − | > Bazaar - Ok
| + | |
| − | > Tar tool - Ok
| + | |
| − | > Unzip tool - Ok
| + | |
| − | > Unrar tool - Ok
| + | |
| − | > GIT - is missing
| + | |
| − | > Make - Ok
| + | |
| − | > cMake - Ok
| + | |
| − | > path tool - Ok
| + | |
| − | > Autotools - Ok
| + | |
| − | | + | |
| − | Perhaps an enhancement to the above would be to show the "build" and "install" status of each.
| + | |
| − | | + | |
| − | '''9. Other behavior'''
| + | |
| − | | + | |
| − | bake.py handles control-C signal to abort the build.
| + | |
| − | | + | |
| − | Bake trusts the standard tools to manage dirty builds and avoid building things unnecessarily. If the user wants, it is possible to call bake uninstall to remove the built code and start a clean build.
| + | |
| − | | + | |
| − | '''10. bake.py handling of ns-3 enabled_modules'''
| + | |
| − | | + | |
| − | For ns-3.17, there is no special handling of this configure options; existing ns-3 mechanisms continue to work (e.g. .ns3rc). It is still an open question of what more to do (if any) for this issue.
| + | |
| − | | + | |
| − | '''11. ns-3's scratch directory'''
| + | |
| − | | + | |
| − | We propose to keep this as is for now. Over time, it may be that we need to provide something similar to scratch/ at the bake level, although it is possible that some of the magic of waf will be replaced by the more explicit use of environment variables along with autotools/Makefile skeletons as well.
| + | |
| − | | + | |
| − | waf may need to learn of these new (default) install locations.
| + | |
| − | | + | |
| − | '''12. DCE'''
| + | |
| − | | + | |
| − | bake.py provides access to ns-3 DCE with at least one well-documented example (iperf) of how to use real application code within an ns-3 program. We focus on user-mode applications for ns-3.17 and kernel-mode for the next release. DCE has its own documentation system; we will look into whether its patch for ns-3 can be merged to ns-3-dev or whether it still needs to sit outside of ns-3-dev.
| + | |
| − | | + | |
| − | = Beyond ns-3.17 =
| + | |
| − | | + | |
| − | '''1. external module (deferring to ns-3.18)'''
| + | |
| − | bake.py allows at least one "external" ns-3-based module to be downloaded and built with ns-3. For example, assume there is an external routing protocol module (ns-3-bgp); we may need to make one of these up for the sake of providing an example.
| + | |
| − | | + | |
| − | For ns-3-bgp to build, it needs to link to ns-3 libraries. So, there needs to be some install step in bake to install to a well-known place in the allinone directory, or in the system.
| + | |
| − | | + | |
| − | For a user program in ns-3.17/scratch to use ns-3-bgp, it needs to be able to find the library and headers. So, the install step on ns-3-bgp should leave the installation in a well-known place by ns-3's waf.
| + | |
| − | | + | |
| − | So, the goal here would be:
| + | |
| − | | + | |
| − | ./bake.py configure -e ns-3-bgp
| + | |
| − | ./bake.py download
| + | |
| − | ./bake.py build # builds ns-3, then builds ns-3-bgp
| + | |
| − | | + | |
| − | Then, user program in ns-3.17/scratch/ can include the module without specifying the full path to ns-3-bgp; i.e., waf may need to become smarter to find these headers and libraries.
| + | |
| − | | + | |
| − | '''Other post-3.16 goals:''' To be determined.
| + | |
1. (software distribution tool) Enable a more decentralized development/release/maintenance environment for ns-3 third-party developers.
1.1 Split out some of existing ns-3-dev into a smaller "core" and additional "extensions". This should also reduce build time for users.
1.2 Provide straightforward way for a third-party developer to maintain their own ns-3 extension and make it available to ns-3 users.
2. (development tool) Help developers better manage multiple builds.
Several (DTN, epidemic routing, routes mobility model, NSC, perhaps others ...) will become available only through the bake tool; large modules will no longer be merged to ns-3-dev but will need to be selected and managed by the bake tool.
This has implications for how we manage Doxygen, Sphinx documentation, the scratch directory, waf, etc.
This means reconciling the existing .ns3rc and bake way of controlling the build.
It has been previously discussed to break apart ns-3-dev into separate repositories going forward, with a completely flat directory structure. This is for further study beyond ns-3.22.
Trying to integrate the above with binary package distribution (.deb, .rpm, etc.) is out of scope for ns-3.22.
