Using Python to Run ns-3

Python bindings allow the C++ code in ns-3 to be called from Python.

This chapter shows you how to create a Python script that can run ns-3 and also the process of creating Python bindings for a C++ ns-3 module.

Introduction

Python bindings provide support for importing ns-3 model libraries as Python modules. Coverage of most of the ns-3 C++ API is provided. The intent has been to allow the programmer to write complete simulation scripts in Python, to allow integration of ns-3 with other Python tools and workflows. The intent is not to provide a different language choice to author new ns-3 models implemented in Python.

Python bindings for ns-3 use a tool called PyBindGen (https://github.com/gjcarneiro/pybindgen) to create Python modules from the C++ libraries built by Waf. The Python bindings that PyBindGen uses are maintained in a bindings directory in each module, and must be maintained to match the C++ API of that ns-3 module. If the C++ API changes, the Python bindings file must either be modified by hand accordingly, or the bindings must be regenerated by an automated scanning process.

If a user is not interested in Python, he or she may disable the use of Python bindings at Waf configure time. In this case, changes to the C++ API of a provided module will not cause the module to fail to compile.

The process for automatically generating Python bindings relies on a toolchain involving a development installation of the Clang compiler, a program called CastXML (https://github.com/CastXML/CastXML), and a program called pygccxml (https://github.com/gccxml/pygccxml). The toolchain can be installed using the ns-3 bake build tool.

An Example Python Script that Runs ns-3

Here is some example code that is written in Python and that runs ns-3, which is written in C++. This Python example can be found in examples/tutorial/first.py:

import ns.applications
import ns.core
import ns.internet
import ns.network
import ns.point_to_point

ns.core.LogComponentEnable("UdpEchoClientApplication", ns.core.LOG_LEVEL_INFO)
ns.core.LogComponentEnable("UdpEchoServerApplication", ns.core.LOG_LEVEL_INFO)

nodes = ns.network.NodeContainer()
nodes.Create(2)

pointToPoint = ns.point_to_point.PointToPointHelper()
pointToPoint.SetDeviceAttribute("DataRate", ns.core.StringValue("5Mbps"))
pointToPoint.SetChannelAttribute("Delay", ns.core.StringValue("2ms"))

devices = pointToPoint.Install(nodes)

stack = ns.internet.InternetStackHelper()
stack.Install(nodes)

address = ns.internet.Ipv4AddressHelper()
address.SetBase(ns.network.Ipv4Address("10.1.1.0"), ns.network.Ipv4Mask("255.255.255.0"))

interfaces = address.Assign (devices);

echoServer = ns.applications.UdpEchoServerHelper(9)

serverApps = echoServer.Install(nodes.Get(1))
serverApps.Start(ns.core.Seconds(1.0))
serverApps.Stop(ns.core.Seconds(10.0))

echoClient = ns.applications.UdpEchoClientHelper(interfaces.GetAddress(1), 9)
echoClient.SetAttribute("MaxPackets", ns.core.UintegerValue(1))
echoClient.SetAttribute("Interval", ns.core.TimeValue(ns.core.Seconds (1.0)))
echoClient.SetAttribute("PacketSize", ns.core.UintegerValue(1024))

clientApps = echoClient.Install(nodes.Get(0))
clientApps.Start(ns.core.Seconds(2.0))
clientApps.Stop(ns.core.Seconds(10.0))

ns.core.Simulator.Run()
ns.core.Simulator.Destroy()

Running Python Scripts

waf contains some options that automatically update the python path to find the ns3 module. To run example programs, there are two ways to use waf to take care of this. One is to run a waf shell; e.g.:

$ ./waf shell
$ python examples/wireless/mixed-wireless.py

and the other is to use the –pyrun option to waf:

$ ./waf --pyrun examples/wireless/mixed-wireless.py

To run a python script under the C debugger:

$ ./waf shell
$ gdb --args python examples/wireless/mixed-wireless.py

To run your own Python script that calls ns-3 and that has this path, /path/to/your/example/my-script.py, do the following:

$ ./waf shell
$ python /path/to/your/example/my-script.py

Caveats

Python bindings for ns-3 are a work in progress, and some limitations are known by developers. Some of these limitations (not all) are listed here.

Incomplete Coverage

First of all, keep in mind that not 100% of the API is supported in Python. Some of the reasons are:

1. some of the APIs involve pointers, which require knowledge of what kind of memory passing semantics (who owns what memory). Such knowledge is not part of the function signatures, and is either documented or sometimes not even documented. Annotations are needed to bind those functions;
2. Sometimes a unusual fundamental data type or C++ construct is used which is not yet supported by PyBindGen;
3. GCC-XML does not report template based classes unless they are instantiated. (Note: Need to confirm this limitation still exists with CastXML)

Most of the missing APIs can be wrapped, given enough time, patience, and expertise, and will likely be wrapped if bug reports are submitted. However, don’t file a bug report saying “bindings are incomplete”, because we do not have manpower to complete 100% of the bindings.

Conversion Constructors

Conversion constructors are not fully supported yet by PyBindGen, and they always act as explicit constructors when translating an API into Python. For example, in C++ you can do this:

Ipv4AddressHelper ipAddrs;
ipAddrs.SetBase ("192.168.0.0", "255.255.255.0");
ipAddrs.Assign (backboneDevices);

In Python, for the time being you have to do:

ipAddrs = ns.internet.Ipv4AddressHelper()
ipAddrs.SetBase(ns.network.Ipv4Address("192.168.0.0"), ns.network.Ipv4Mask("255.255.255.0"))
ipAddrs.Assign(backboneDevices)

CommandLine

CommandLine::AddValue() works differently in Python than it does in ns-3. In Python, the first parameter is a string that represents the command-line option name. When the option is set, an attribute with the same name as the option name is set on the CommandLine() object. Example:

NUM_NODES_SIDE_DEFAULT = 3

cmd = ns3.CommandLine()

cmd.NumNodesSide = None
cmd.AddValue("NumNodesSide", "Grid side number of nodes (total number of nodes will be this number squared)")

cmd.Parse(argv)

[...]

if cmd.NumNodesSide is None:
    num_nodes_side = NUM_NODES_SIDE_DEFAULT
else:
    num_nodes_side = int(cmd.NumNodesSide)

Tracing

Callback based tracing is not yet properly supported for Python, as new ns-3 API needs to be provided for this to be supported.

Pcap file writing is supported via the normal API.

ASCII tracing is supported since ns-3.4 via the normal C++ API translated to Python. However, ASCII tracing requires the creation of an ostream object to pass into the ASCII tracing methods. In Python, the C++ std::ofstream has been minimally wrapped to allow this. For example:

ascii = ns3.ofstream("wifi-ap.tr") # create the file
ns3.YansWifiPhyHelper.EnableAsciiAll(ascii)
ns3.Simulator.Run()
ns3.Simulator.Destroy()
ascii.close() # close the file

There is one caveat: you must not allow the file object to be garbage collected while ns-3 is still using it. That means that the ‘ascii’ variable above must not be allowed to go out of scope or else the program will crash.

Working with Python Bindings

Python bindings are built on a module-by-module basis, and can be found in each module’s bindings directory.

Overview

The python bindings are generated into an ‘ns’ namespace. Examples:

from ns.network import Node
n1 = Node()

or

import ns.network
n1 = ns.network.Node()

The best way to explore the bindings is to look at the various example programs provided in ns-3; some C++ examples have a corresponding Python example. There is no structured documentation for the Python bindings like there is Doxygen for the C++ API, but the Doxygen can be consulted to understand how the C++ API works.

Python Bindings Workflow

The process by which Python bindings are handled is the following:

1. Periodically a developer uses a CastXML (https://github.com/CastXML/CastXML) based API scanning script, which saves the scanned API definition as bindings/python/ns3_module_*.py files or as Python files in each modules’ bindings directory. These files are kept under version control in the main ns-3 repository;
2. Other developers clone the repository and use the already scanned API definitions;
3. When configuring ns-3, pybindgen will be automatically downloaded if not already installed. Released ns-3 tarballs will ship a copy of pybindgen.

If something goes wrong with compiling Python bindings and you just want to ignore them and move on with C++, you can disable Python with:

$ ./waf configure --disable-python ...

To add support for modular bindings to an existing or new ns-3 module, simply add the following line to its wscript build() function:

bld.ns3_python_bindings()

One must also provide the bindings files (usually by running the scanning framework).

Regenerating the Python bindings

ns-3 will fail to successfully compile the Python bindings if the C++ headers are changed and no longer align with the stored Python bindings. In this case, the developer has two main choices: 1) disable Python as described above, or 2) update the bindings to align with the new C++ API.

Process Overview

ns-3 has an automated process to regenerate Python bindings from the C++ header files. The automated process is only semi-automated at the moment (ns-3.27) because we are in the midst of transition to new tools. The current process is outlined below. In short, the process currently requires the following steps.

1. Prepare the system for scanning by installing the prerequisites, including a development version of clang, the CastXML package, and pygccxml.
2. Perform a 64-bit scan of the module or modules of interest
3. Create the 32-bit bindings file from the 64-bit scan output

Installing a clang development environment

Make sure you have a development version of the clang compiler installed on your system. This can take a long time to build from source. Linux distributions provide binary library packages such as clang-dev or clang-devel. The version should not be too important; version 3.8 is known to work. Note that there is a problem with the Ubuntu package installation of clang-dev; see the Installation wiki page for details on how to fix using some symlinks.

Installing other prerequisites

cxxfilt is a new requirement, typically installed using pip; e.g.

sudo pip install cxxfilt

See also the wiki for installation notes for your system.

Set up a bake build environment

Try the following commands:

.. sourcecode:: bash

$ cd bake $ export BAKE_HOME=`pwd` $ export PATH=$PATH:$BAKE_HOME/build/bin $ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$BAKE_HOME/build/lib $ export PYTHONPATH=$PYTHONPATH:$BAKE_HOME/build/lib $ mkdir -p build/lib

Configure

Perform a configuration at the bake level:

.. sourcecode:: bash

$ ./bake.py configure -e ns-3-dev -e pygccxml-1.9.1

The output of bake show should show something like this:

.. sourcecode:: bash

$ ./bake.py show

Should say:

.. sourcecode:: text

– System Dependencies –

> clang-dev - OK > g++ - OK > libxml2-dev - OK > pygoocanvas - OK > pygraphviz - OK > python-dev - OK > qt - OK > setuptools - OK

Download

Try the following command:

.. sourcecode:: bash

$ ./bake.py download

>> Searching for system dependency python-dev - OK >> Searching for system dependency clang-dev - OK >> Searching for system dependency g++ - OK >> Searching for system dependency setuptools - OK >> Searching for system dependency pygoocanvas - OK >> Searching for system dependency pygraphviz - OK >> Searching for system dependency qt - OK >> Downloading castxml - OK >> Downloading netanim - OK >> Downloading pygccxml-1.9.1 - OK >> Downloading pygccxml - OK >> Downloading pybindgen - OK >> Downloading ns-3-dev - OK

Build

Try the following commands:

.. sourcecode:: text

$ mkdir -p build/lib $ ./bake.py build

It should fail on the ns-3 bindings complilation.

The output of ‘./waf configure’ can be inspected to see if Python API scanning support is enabled:

.. sourcecode:: text

Python API Scanning Support : enabled

It may say something like this, if the support is not active:

.. sourcecode:: text

Python API Scanning Support : not enabled (Missing ‘pygccxml’ Python module)

In this case, the user must take steps to install castxml and pygccxml; castxml binary must be in the shell’s path, and pygccxml must be in the Python path.

Rescan 64-bit

It is important that you scan the failing module with ‘–no32bit-scan’; e.g.:

.. sourcecode:: bash

$ cd source/ns-3-dev $ ./waf –apiscan=wifi –no32bit-scan

Generate 32-bit

Once the 64-bit bindings are fixed, if you are a maintainer and need to generate the 32-bit equivalent:

1. copy the newly scanned LP64.py file(s) to the ILP32.py filename
2. on the ILP32.py file(s), perform global substitition of unsigned long with unsigned long long

Linux (64-bit, as most modern installations use) and MacOS use different data models, as explained here: https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.3.0/com.ibm.zos.v2r3.cbcpx01/datatypesize64.htm

Linux uses the LP64 model, and MacOS (as well as 32-bit Linux) use the ILP32 model. Users will note that there are two versions of bindings files in each ns-3 module directory; one with an ILP32.py suffix and one with an LP64.py suffix. Only one is used on any given platform. The main difference is in the representation of the 64 bit integer type as either a ‘long’ (LP64) or ‘long long’ (ILP32).

As of ns-3.27, the CastXML framework can generate LP64 bindings by passing the --no32bit-scan flag. However, the framework does not automatically generate 32bit scans at the moment. Instead, users must manually generate the ILP32.py equivalents by taking all instances of ‘unsigned long’ in the bindings file and converting them to ‘unsigned long long’, such as:

-    cls.add_instance_attribute('nMarkedBytes', 'std::map< std::string, unsigned long >', is_const=False)
+    cls.add_instance_attribute('nMarkedBytes', 'std::map< std::string, unsigned long long >', is_const=False)

In summary, to generate LP64 bindings for Linux 64-bit systems, it is sufficient to call (e.g. for the core module):

.. sourcecode:: bash

$ ./waf –apiscan=core –no32bit-scan

To generate ILP32 bindings, one first must generate the LP64.py file as above, and then copy the file to be named with an ILP32.py suffix, and then hand-edit that file, replacing all instances of ‘unsigned long’ with ‘unsigned long long’. ns-3 maintainers are working to better automate this process for future releases.

Generating bindings on MacOS

In principle, this should work (and should generate the 32-bit bindings). However, it is untested and we are not sure what instructions to offer. We would welcome suggestions on how to enable scanning for MacOS.

Organization of the Modular Python Bindings

The src/<module>/bindings directory may contain the following files, some of them optional:

• callbacks_list.py: this is a scanned file, DO NOT TOUCH. Contains a list of Callback<...> template instances found in the scanned headers;
• modulegen__gcc_LP64.py: this is a scanned file, DO NOT TOUCH. Scanned API definitions for the GCC, LP64 architecture (64-bit)
• modulegen__gcc_ILP32.py: this is a scanned file, DO NOT TOUCH. Scanned API definitions for the GCC, ILP32 architecture (32-bit)
• modulegen_customizations.py: you may optionally add this file in order to customize the pybindgen code generation
• scan-header.h: you may optionally add this file to customize what header file is scanned for the module. Basically this file is scanned instead of ns3/<module>-module.h. Typically, the first statement is #include “ns3/<module>-module.h”, plus some other stuff to force template instantiations;
• module_helpers.cc: you may add additional files, such as this, to be linked to python extension module, but they have to be registered in the wscript. Look at src/core/wscript for an example of how to do so;
• <module>.py: if this file exists, it becomes the “frontend” python module for the ns3 module, and the extension module (.so file) becomes _<module>.so instead of <module>.so. The <module>.py file has to import all symbols from the module _<module> (this is more tricky than it sounds, see src/core/bindings/core.py for an example), and then can add some additional pure-python definitions.

More Information for Developers

If you are a developer and need more information on ns-3‘s Python bindings, please see the Python Bindings wiki page.