Python3

From Nsnam
Revision as of 21:25, 3 June 2015 by Pdbarnes (Talk | contribs) (Tasks: raise upload.py)

Jump to: navigation, search

Main Page - Current Development - Developer FAQ - Tools - Related Projects - Project Ideas - Summer Projects

Installation - Troubleshooting - User FAQ - HOWTOs - Samples - Models - Education - Contributed Code - Papers

This page is tracking Python3 compatibility for ns-3 and related projects. As of ns-3.23 (May 2015), only Python2 (2.7 series) is supported.

Goals

Make ns-3 and related projects (bake build system, pyviz visualizer) compatible with both Python 2.7 and 3.4.

Tasks

The following ports for basic ns-3 operation need to be accomplished (if found incompatible with Python3):

  1. All scripts for the waf build system (waf-tools, wutils.py, etc.)
  2. All ns-3 python programs (e.g. first.py)
  3. src/visualizer
  4. The test.py test runner program
  5. All python programs in the documentation directories (e.g. conf.py)
  6. All scripts in ns-3-allinone (build.py dist.py download.py)
  7. src/create-module.py
  8. utils.py
  9. utils/check-style.py
  10. utils/grid.py
  11. utils/python-unit-tests.py

Pdbarnes (talk) What about the Rietveld upload.py script used for code review? This is their issue #486

Also, the bake build system must be ported.

Also, the Python API scanning process needs to be upgraded. This relies on third party tools (particularly pygccxml and pybindgen) that need to be investigated.

Status

The most recent code is maintained at http://code.nsnam.org/tomh/ns-3-dev-python3/ until we merge it. A summer project from 2014 by Siddharth Sankurtar has accomplished much of the initial port of ns-3 portions (above) to Python-3.3.5; the above repository includes Siddharth's previous work on this.

One of the goals of ns-3.24 release is to at least get to compatibility of the core of ns-3, even if bake and python scanning does not yet work.

There are three main remaining issues (patches requested for 1) and 2)):

1. Something has broken in moving from Python 3.3 to Python 3.4

 _pickle.PicklingError: Can't pickle <class 'waflib.Node.Context.__init__.<locals>.node_class'>: attribute lookup   
 Context.__init__.<locals>.node_class on waflib.Node failed

and

     os.chmod(dst, 0600)
                    ^
 SyntaxError: invalid token

2. In python 3.3, the following issue was encountered and not debugged yet.

During the first build phase, waf generates signatures for every task. Upon successful completion of the task, the signature of the task is stored into a hidden file in the build directory. The task signatures are serialized using the Pickle library.

Following builds after this would load the task signatures from the hidden file. For every task, it generates its signature and checks if it is present in the dictionary of the loaded task signatures. In case of Python3, for some tasks, the newly generated signature fails to match with the loaded signature, causing a KeyError and consequently a re-building of that task. The issue possibly lies in the signature generation in Python3.

3. Siddharth has written some test programs to test waf and test.py themselves; do we want to add them to a test directory? See:

  • utils/tests/TestBase.py
  • utils/tests/test-test.py
  • utils/tests/test-waf.py

Also, beyond basic ns-3, we need volunteers to work on these additional issues:

  • Siddharth Santurkar has volunteered to port bake
  • We don't have anyone working on Python API scanning toolchain (please contact us if you want to work on this)

Testing

The latest ns-3 code is available at http://code.nsnam.org/tomh/ns-3-dev-python3

Python3 can be tested on most Linux systems using the virtualenv system.

If you have 'pip' installed, you can install virtualenv as follows:

 $ sudo pip install virtualenv

Then, the following steps are recommended:

 $ mkdir my_project_folder && cd $_
 $ hg clone http://code.nsnam.org/ns-3-allinone
 $ cd ns-3-allinone
 $ /usr/bin/python2.7 download.py
 $ hg clone http://code.nsnam.org/tomh/ns-3-dev-python3
 $ virtualenv -p /usr/bin/python3 venv
 $ source venv/bin/activate

Your prompt should now look like this, prepended with '(venv)':

 (venv)buildslave@slave-ubuntu-15:~/my_project_folder/ns-3-allinone$

Now you are in a special environment for which any calls to 'python' will use the python3 version. To see this, try calling:

 (venv)buildslave@slave-ubuntu-15:~/my_project_folder/ns-3-allinone$ python -V

To drop out of this virtualenv, type:

 $ deactivate

You may also consider to run the test-test.py and test-waf.py programs (which unit test the test.py and waf programs and their options) found in ``utils/tests/`` directory.