GSOC2023ns3-aiFinalReport
Main Page - Roadmap - Summer Projects - Project Ideas - Developer FAQ - Tools - Related Projects
HOWTOs - Installation - Troubleshooting - User FAQ - Samples - Models - Education - Contributed Code - Papers
Back to GSOC2023ns3-ai (page containing my weekly updates, not the final report)
Project Overview
- Project Name: ns3-ai enhancements
- Student: Muyuan Shen
- Mentors: Collin Brady and Hao Yin
Project Goals
The main focus of this project is to optimize performance and improve usability of the ns3-ai module, which facilitates the connection between ns-3 and Python-based ML frameworks using shared memory.
To accomplish this goal, the project will introduce additional APIs that support data structures such as vector in shared memory IPC. This will effectively reduce the required interaction between C++ and Python, resulting in improved performance. Also, the project will integrate Gymnasium API like ns3-gym's but has a shared-memory-based backend, to turn ns-3 into a environment that agents can efficiently and seamlessly interact with. In addition, the project will enhance the existing examples, documentation and tutorials, while also integrating new examples that cover scenarios like Multi-BSS in VR. By doing so, users will have more comprehensive resources at their disposal. Furthermore, the project aims to provide examples utilizing pure C++-based ML frameworks. This will offer researchers more options for integrating with ML.
The overall aim of the project is to expand and accelerate the capabilities of the ns3-ai module, enabling users to simulate and analyze network related algorithms with enhanced efficiency and flexibility.
Merge Requests and Commits
Throughout the project, my development is based my improvements branch of ns3-ai. The improvements branch originates from the cmake branch, because I was fixing problems about cmake compatibility before GSoC. So, I created a single MR that contain all my works to be merged into cmake branch. In this MR, there are 120+ commits by me, with author name 'ShenMuyuan' or 'Mu-YuanShen' or 'eicsmy'. The cmake branch will be merged to upstream by my mentor.
| No. | Name | Status |
|---|---|---|
| [1] | merge to cmake branch | Open |
Project Details
All links to my repository below belongs to my last commit to improvements branch.
Community Bonding Period
During community bonding period, I started bi-weekly meetings with my mentors and we decided on the project plan, which is prioritizing the development of new interfaces, than develop more examples & enhance documentations.
There are two new interfaces, including vector interface (later, we called it vector-based message interface, as it shared some fundamentals with the struct-based message interface) and Gym interface. Also, we talked about some details of new examples like LTE-handover and Multi-BSS.
I also read the ns3-ai code thoroughly to understand its IPC principles and learned some reinforcement learning basics.
Phase 1
std::vector support
To add std::vector into shared memory is not easy with ns3-ai's original design, because Python's ctypes library does not provide STL templates support (it can only support C structures and functions). In order to support vector, I refactored the original model completely, replacing ctypes with Boost C++ library which is more flexible for interprocess communication. My works include:
- Utilized Boost's boost::interprocess::managed_shared_memory to store data (as well as synchronization variables) in shared memory. This shared segment can be used for data transmission between C++ and Python. The two directions, C++-to-Python and Python-to-C++, occupies two different regions in shared memory. It also supports custom memory allocator for STL, a instance of boost::interprocess::allocator, which ensures that when STL allocates new memory, that memory is come from the shared memory rather than other heap memory.
- The shared memory creation can be found in the constructor of Ns3AiMsgInterfaceImpl: code
- Developed spinlock-based semaphore to synchronize reads & writes operations in shared memory. The original synchronization method works, but the "version number" concept and the "control block" data structures may cause confusion and distraction for beginners. Also, the "version number" is just a complex implementation of the well-known semaphore. To improve ease of use and enhance code readability, I created a semaphore that only spins but does not sleep while waiting based on Boost's semaphore. It has performance comparable to the original with better readability and usability.
- Built the vector-based interface with multiple configurable options. The vector interface is in parallel with the struct interface in terms of creation and usage, and there is an attribute that users can set in early code in order to choose one of the interfaces. If the vector interface is chosen, the C++-to-Python and Python-to-C++ vectors are created in shared memory and will contain no elements. It requires users to call resize or push_back to adjust their length before use. Another attribute is whether the interface handles simulation end. If that attribute is set, the interface will perform a simple protocol to notify Python side when C++ side simulation finishes. Other configurable attributes include memory segment size and names of objects constructed in shared memory.
- Provided Python binding boilerplate code in examples. Python side accesses the shared memory and the objects in it (vectors or structs) via C++ functions exposed to Python. The exposure of C++ class functions and members is achieved with Pybind11, a lightweight python binding library. The C++ binding code, linked with Pybind11, is compiled into dynamically-linked library that Python can import as a module. Because the C++ side interface is template-based and Python does not support template natively, the Python binding module needs to be separately generated for every program (the creation is done by a cmake target dependency so it's seamless). Although the binding contains many lines of C++ code and is difficult to write from scratch, users can modify from an existing binding code to generate Python binding modules quickly, and I provide many boilerplate on that (the *_py.cc files in all examples).
- Some of the example boilerplate code: binding code for struct-based message interface in A-Plus-B example, binding code for vector-based message interface in Multi-BSS example
Gymnasium API
The Gymnasium API for ns3-ai is aimed to be based on shared memory rather than sockets communication, which can provide faster data exchange than ns3-gym does. While many of the Gym interface code is from ns3-gym's repository, I made some substantial changes in order for it to have a shared memory backend. My works include:
- Modified OpenGymInterface to use Ns3AiMsgInterface for IPC. OpenGymInterface is created by ns3-gym developers, providing code to create Gym-compatible environments in ns-3. It contains functions to get state or action spaces, observe the environment in ns-3 and execute the actions (maybe changing parameters in simulation). Those function use callbacks registered by OpenGymEnv at runtime. To make callbacks work well, custom environment must inherit from OpenGymEnv and implement the class methods such as GetActionSpace, GetObservationSpace, GetObservation and ExecuteActions. All states and actions are serialized by Google's Protocol Buffers and then transmitted and de-serialized by the peer. What I did is changing the ZeroMQ socket's send & receive functions to Ns3AiMsgInterface's send & receive functions, and ensuring that Ns3AiMsgInterface is properly initialized. The underlying message interface for transmitting serialized messages is struct-based. The struct contains a buffer (uint8_t array) and its capacity.
- Example of my changed part: before (in ns3-gym's repo), after (in my ns3-ai repo)
- Initialization of Ns3AiMsgInterface: code
- Note: in the above configuration, handling finish is set to false because the protocol of notifying Python side that C++ side has finished is unnecessary for Gym. Gym interface has its own protocol for handling finish, which is NotifySimulationEnd on C++ side and then 'done' becoming true when Python steps.
- Created Python binding for accessing the shared structure containing serialized message string. Binding that structure containing array is similar to binding a common structure, except that the array is specially treated to convert its contents to Python's memoryview. With memoryview, Python side can read and write to the array seamlessly, like what you can do in C++ with std::array.
- Obtaining the memoryview in binding: code
- Note: different length of array must have different memoryview object for Python to deal with. In the above code, get_buffer returns the buffer that is actually used (for reading), while get_buffer_full returns the buffer that has the full length (for writing). Example usage in Ns3Env (the Python side Gym environment created with gym.make): reading and writing