GSOC2023ns3-ai - Revision history

Muyuan at 13:49, 7 September 2023

2023-09-07T13:49:00Z

← Older revision		Revision as of 13:49, 7 September 2023
Line 9:		Line 9:
	* '''Mentors:''' Collin Brady and Hao Yin		* '''Mentors:''' Collin Brady and Hao Yin
	* '''Google page:''' https://summerofcode.withgoogle.com/programs/2023/projects/A4KZ7dxo		* '''Google page:''' https://summerofcode.withgoogle.com/programs/2023/projects/A4KZ7dxo
			* '''Repository:''' https://github.com/ShenMuyuan/ns3-ai/tree/improvements
			* '''Final report:''' [[GSOC2023ns3-aiFinalReport]]

	== Project ~~Goals~~ ==		== About the Project ==

	The proposed project ~~aims~~ to enhance the ns3-ai module, which ~~provides interfaces~~ between ns-3 and Python-based ML frameworks using shared memory~~, with a~~ focus ~~on performance optimization and expanding the range~~ of ~~supported data structures.~~		The objective of this proposed project is to enhance the ns3-ai module, which facilitates the connection between ns-3 and Python-based ML frameworks using shared memory. The main focus of this enhancement is to optimize performance and improve usability.
	~~To achieve~~ this~~, the project will introduce APIs for additional data structures like vector and string in shared memory IPC~~ to ~~reduce the interaction between C++~~ and Python. Additionally, the project will provide examples demonstrating how to implement ML algorithms within ns-3 using C++ and open-source frameworks such as TensorFlow and PyTorch. The project will also improve ~~the current examples and documentation and integrate new examples, such as LTE handover.~~
	~~Overall, the project aims to expand and accelerate the capabilities of the ns3-ai module, enabling users to simulate and analyze large-scale networks with greater efficiency and flexibility~~.

	~~== Repository ==~~		To accomplish this goal, the project will introduce additional APIs that support data structures such as vector in shared memory IPC. This will effectively reduce the required interaction between C++ and Python, resulting in improved performance. Also, the project will integrate Gymnasium API like ns3-gym's but has a shared-memory-based backend, to turn ns-3 into an environment that agents can efficiently and seamlessly interact with.

	~~https://github~~.~~com/ShenMuyuan/~~ns3-ai~~/tree/improvements~~		In addition, the project will enhance the existing examples, documentation and tutorials, while also integrating new examples that cover scenarios like Multi-BSS in VR. By doing so, users will have more comprehensive resources at their disposal.

			Furthermore, the project aims to provide examples utilizing pure C++-based ML frameworks. This will offer researchers more options for integrating with ML. The overall aim of the project is to expand and accelerate the capabilities of the ns3-ai module, enabling users to simulate and analyze network related algorithms with enhanced efficiency and flexibility.

	== About Me ==		== About Me ==

	~~=== Education ===~~

	As a junior at Huazhong University of Science and Technology, I am majoring in electronic engineering. I am proud to be a member of the Undergraduate Program for Advanced Project-based Information Science Education, also known as the Seed Class, and currently serve as the class monitor. Additionally, I am a project leader in the Dian group, where I engage in extracurricular technical projects.		As a junior at Huazhong University of Science and Technology, I am majoring in electronic engineering. I am proud to be a member of the Undergraduate Program for Advanced Project-based Information Science Education, also known as the Seed Class, and currently serve as the class monitor. Additionally, I am a project leader in the Dian group, where I engage in extracurricular technical projects.
	In terms of relevant coursework, I have excelled in network programming through courses such as C programming language and computer network, both of which I achieved a perfect grade point of 4.0. These courses have equipped me with a strong foundation in network programming, which I believe will enable me to contribute effectively to relevant projects.		In terms of relevant coursework, I have excelled in network programming through courses such as C programming language and computer network, both of which I achieved a perfect grade point of 4.0. These courses have equipped me with a strong foundation in network programming, which I believe will enable me to contribute effectively to relevant projects.
	I am a motivated and skilled undergraduate student with a passion for network programming and a track record of academic excellence.		I am a motivated and skilled undergraduate student with a passion for network programming and a track record of academic excellence.

	~~=== Experience with ns-3 ===~~

	During my academic journey, I have had the opportunity to explore computer networking through labs and projects. In particular, in the labs for the computer networking course, I gained valuable insights into how different parameters, such as the number of STAs, CW range, and packet arrival rate, can impact network throughput in the WiFi DCF protocol.		During my academic journey, I have had the opportunity to explore computer networking through labs and projects. In particular, in the labs for the computer networking course, I gained valuable insights into how different parameters, such as the number of STAs, CW range, and packet arrival rate, can impact network throughput in the WiFi DCF protocol.
Line 36:		Line 34:
	= Milestones =		= Milestones =

	Based on my ~~[https://diangroup.feishu.cn/docx/PSK9d18KLoLvXCxmGIkcbhXTn5b~~ proposal], I divide my project into two phases, listed below.		Based on my proposal, I divide my project into two phases, listed below.

	== Phase one (before midterm evaluation) ==		== Phase one (before midterm evaluation) ==
Line 280:		Line 278:
	# Finish the last C++-based ML example (RL-TCP using PyTorch C++ API)		# Finish the last C++-based ML example (RL-TCP using PyTorch C++ API)
	# Update vector-based message interface's benchmarking result.		# Update vector-based message interface's benchmarking result.

			== Week 14 (Aug 28 - Aug 3) ==

			=== Achievements ===

			# Finished RL-TCP example using pure C++ interface
			# Benchmarked pure C++ (pure C++ vs msg interface in terms of processing time) and vector-based interface (vector vs struct in terms of data transmission time)
			#* Pure C++ is twice faster than msg interface, due to removal of interprocess communication.
			#* Unfortunately, vector-based message interface is slower (1.2x in CPU cycle count) than struct-based interface. Possibly due to slow access of vector on Python side.
			# Updated many documents for PR.

			=== Problems ===

			# As mentioned above, vector interface is slow. In the future, we may need to integrate C++-based linear algebra libraries like Eigen, which has faster Python bindings than pybind11's std::vector binding, provided by some open source projects like eigenpy.

Muyuan at 02:20, 29 August 2023

2023-08-29T02:20:57Z

← Older revision		Revision as of 02:20, 29 August 2023
Line 263:		Line 263:
	# Adjust the parameters in RL algorithm to meet VR requirements on throughput		# Adjust the parameters in RL algorithm to meet VR requirements on throughput
	# Benchmarking (compare running time with previous interface)		# Benchmarking (compare running time with previous interface)

			== Week 13 (Aug 21 - Aug 27) ==

			=== Achievements ===

			# Finished a simple example using C++-based ML framework TensorFlow.
			# Continue modifying Multi-BSS parameters to meet VR requirements.

			=== Problems ===

			# The C++-based example is incomplete, due to the lack of TensorFlow C API as described in their [https://github.com/tensorflow/docs/blob/master/site/en/r1/guide/extend/bindings.md#current-status current-status]. The example will be available when TensorFlow C API provides Gradients and Neural Networks library.
			# The Multi-BSS example still doesn't meet the VR requirements after training, although I changed some parameters. Maybe I have to leave its optimization to future work (after GSoC). The benchmarking is not affected.

			=== Todo next week ===

			# Finish the last C++-based ML example (RL-TCP using PyTorch C++ API)
			# Update vector-based message interface's benchmarking result.

Muyuan at 07:37, 25 August 2023

2023-08-25T07:37:52Z

← Older revision		Revision as of 07:37, 25 August 2023
Line 228:		Line 228:
	# Finish multi-bss example. My work will include making it compile with the latest ns-3, porting it to ns3-ai's new interface and changing some directory structure (move the tgax code under src to contrib).		# Finish multi-bss example. My work will include making it compile with the latest ns-3, porting it to ns3-ai's new interface and changing some directory structure (move the tgax code under src to contrib).

	== Week 10 (Aug 7 - Aug 13) ==		== Week 11 (Aug 7 - Aug 13) ==

	=== Achievements ===		=== Achievements ===
Line 245:		Line 245:
	# Complete the Multi-BSS example's code and do some benchmark.		# Complete the Multi-BSS example's code and do some benchmark.

	== Week 10 (Aug 14 - Aug 20) ==		== Week 12 (Aug 14 - Aug 20) ==

	=== Achievements ===		=== Achievements ===

Muyuan: /* Finishing new examples and benchmarking test */

2023-08-25T03:16:56Z

Finishing new examples and benchmarking test

← Older revision		Revision as of 03:16, 25 August 2023
Line 64:		Line 64:
	=== Finishing new examples and benchmarking test ===		=== Finishing new examples and benchmarking test ===

	~~TODO~~		Finish some new examples using Gym interface and vector-based message interface. Compare Gym interface's performance with ns3-gym,
			and compare vector-based message interface's performance with struct-based message interface.

	= Weekly Report =		= Weekly Report =

Muyuan: /* Integration of ns-3 and C++-based ML frameworks */

2023-08-25T03:13:50Z

Integration of ns-3 and C++-based ML frameworks

← Older revision		Revision as of 03:13, 25 August 2023
Line 58:		Line 58:
	=== Integration of ns-3 and C++-based ML frameworks ===		=== Integration of ns-3 and C++-based ML frameworks ===

	~~TODO~~		Apply [https://www.tensorflow.org/api_docs/cc Tensorflow C++ APIs] and [https://pytorch.org/cppdocs/ PyTorch C++ APIs] to examples
			using Python-based ML frameworks. Also, provide Cmake configurations that both works on Linux and macOS, and documentation on building
			& running.

	=== Finishing new examples and benchmarking test ===		=== Finishing new examples and benchmarking test ===

Muyuan at 00:39, 18 August 2023

2023-08-18T00:39:33Z

← Older revision		Revision as of 00:39, 18 August 2023
Line 224:		Line 224:

	# Finish multi-bss example. My work will include making it compile with the latest ns-3, porting it to ns3-ai's new interface and changing some directory structure (move the tgax code under src to contrib).		# Finish multi-bss example. My work will include making it compile with the latest ns-3, porting it to ns3-ai's new interface and changing some directory structure (move the tgax code under src to contrib).

			== Week 10 (Aug 7 - Aug 13) ==

			=== Achievements ===

			# Get familiar with the RL algorithm in multi-bss example, based on [https://www.nsnam.org/tutorials/consortium23/MultiBSS-UW.pdf the slide] and [https://gitlab.com/juanvleonr/ns-3-dev/-/blob/clean-tgax/scratch/tgax-residential.cc?ref_type=heads the code].
			# Finished the Python binding and cmake configuration of multi-bss example, with some problems.
			# Wrote a quick start guide on the ns3-ai interface, and thoroughly updated the READMEs in the repository.

			=== Problems ===

			# The Python script of multi-BSS is incompatible with C++ code, having sightly different data definition.
			# The algorithm should automatically change the CCA, but CAA is not changed (always -82)

			=== Todo next week ===

			# Complete the Multi-BSS example's code and do some benchmark.

			== Week 10 (Aug 14 - Aug 20) ==

			=== Achievements ===

			# Finish the code for Multi-BSS (can run now and the CCA value changes to approximately -70)
			# Updating documents for future release by Hao
			# Enhanced some cmake configurations for better usability
			#* Add protobuf-generate function for protobuf installations that don’t provide it
			#* Change the cmake target from ‘ns3-ai’ to ‘ai’. Now the ns3-ai mudule can be built with ./ns3 build ai. (custom modules cannot be built directly with ./ns3 build if they have a ‘ns3-’ prefix, due to some settings in ./ns3 script)

			=== Problems ===

			# In Multi-BSS example, at 1 min of simulation, VR tpt ≈ 5Mbps, can’t meet requirement (50 Mbps). Occasionally VR delay ≈ 0 and tpt ≈ 1e8, possibly due to statistics error.

			=== Todo next week ===

			# Adjust the parameters in RL algorithm to meet VR requirements on throughput
			# Benchmarking (compare running time with previous interface)

Muyuan at 03:23, 10 August 2023

2023-08-10T03:23:46Z

← Older revision		Revision as of 03:23, 10 August 2023
Line 211:		Line 211:

	# Began developing Multi-BSS example which can demonstrate the usage of vector in message interface.		# Began developing Multi-BSS example which can demonstrate the usage of vector in message interface.

			== Week 10 (July 31 - Aug 6) ==

			=== Achievements ===

			# Update the lte-cqi example to use the msg interface.
			# Working on multi-bss example, based on [https://gitlab.com/juanvleonr/ns-3-dev/-/blob/clean-tgax/scratch/tgax-residential.cc?ref_type=heads Juan's branch of ns-3-dev].

			=== Problems ===

			=== Todo next week ===

			# Finish multi-bss example. My work will include making it compile with the latest ns-3, porting it to ns3-ai's new interface and changing some directory structure (move the tgax code under src to contrib).

Muyuan at 07:12, 30 July 2023

2023-07-30T07:12:53Z

← Older revision		Revision as of 07:12, 30 July 2023
Line 188:		Line 188:

	# Conduct benchmarking of interaction time on RL-TCP example.		# Conduct benchmarking of interaction time on RL-TCP example.

			== Week 9 (July 24 - July 30) ==

			=== Achievements ===

			# Benchmarked RL-TCP example (ns3-gym and ns3-ai's Gym interface version) based on C++-Python interaction time. Interaction time is the transmission time of the byte buffer containing serialized Gym environments or actions. To get accurate interaction time, I use CPU cycle (rdtsc in x86 instructions) rather than clock time. Each saved data is the end CPU cycle of a transmission minus the start CPU cycle of that transmission. The mean and standard deviation of the data are calculated. The result shows that in both C++ to Python and Python to C++ directions, the interaction time of ns3-ai is approximately 15 times shorter than that of ns3-gym.
			#* ns-3 configuration: ./ns3 configure --enable-examples --build-profile=debug
			#* Simulation parameters:
			#** bottleneck_bandwidth=2Mbps
			#** bottleneck_delay=0.01ms
			#** access_bandwidth=10Mbps
			#** access_delay=20ms
			#** duration=100s
			#** time step = 0.1s
			#* Benchmark results:
			#** [https://github.com/ShenMuyuan/urban-pancake/blob/50ad463ee06377342ff83c9954a13cc66792b4d1/ns3ai_benchmark/cpp2py.png C++ to Python transmission time]
			#** [https://github.com/ShenMuyuan/urban-pancake/blob/50ad463ee06377342ff83c9954a13cc66792b4d1/ns3ai_benchmark/py2cpp.png Python to C++ transmission time]

			=== Problems ===

			=== Todo next week ===

			# Began developing Multi-BSS example which can demonstrate the usage of vector in message interface.

Muyuan at 11:48, 29 July 2023

2023-07-29T11:48:57Z

← Older revision		Revision as of 11:48, 29 July 2023
Line 173:		Line 173:
	# Provide some initial benchmark of Gym interface with ns3-gym.		# Provide some initial benchmark of Gym interface with ns3-gym.
	# Do midterm evaluation.		# Do midterm evaluation.

			== Week 8 (July 17 - July 23) ==

			=== Achievements ===

			# '''Successfully finished midterm evaluation. Thank you my mentors Collin and Hao for your guidance!'''
			# Benchmarked the running time of RL-TCP example. In the scenario of 2 nodes with bottleneck_bandwidth=2Mbps, bottleneck_delay=0.01ms, access_bandwidth=10Mbps, access_delay=20ms, I simulated for 1000s and the results shows that ns3-ai is slightly faster then ns3-gym: ns3-ai costs 26 seconds and ns3-gym costs 27 seconds.

			=== Problems ===

			# My mentor suggests that the benchmark doesn't show the advantage of ns3-ai because it uses the total running time rather than C++-Python interaction time. Interaction time is more likely to have a big difference between ns3-ai and ns3-gym because interaction is the place where ns3-ai and ns3-gym differ most. Also, after knowing the C++-Python interaction time and the portion it takes in total time, it's easier to design examples that emphasize the interaction time and better demonstrate the performance of ns3-ai.

			=== Todo next week ===

			# Conduct benchmarking of interaction time on RL-TCP example.

Muyuan at 15:14, 18 July 2023

2023-07-18T15:14:41Z

← Older revision		Revision as of 15:14, 18 July 2023
Line 152:		Line 152:

	# Continue developing other examples using Gym interface.		# Continue developing other examples using Gym interface.

			== Week 6 - 7 (July 3 - July 16) ==

			About interface naming: for clarity, I call the interface that uses Boost shared memory directly (in which users need to define the shared structures or vectors) "msg interface", and the interface that is based on msg interface and provides Gym APIs "Gym interface". The former is low level, requires more coding and has stronger capabilities (such as std::vector sharing), while the latter is high level, easier to code but has limited functionality (RL with Gym).

			=== Achievements ===

			# Due to my mentors' suggestions, I modified the Gym interface so that it provides a base class that users can derive to make their own environment. Basically it is a fork of ns3-gym's interface, but in low level it uses Boost instead of ZeroMQ for interprocess communication.
			# Completed the RL-TCP example using Gym interface & msg interface, and A plus B example using Gym interface.
			# Done refactor of existing code including separating different interfaces in different directories and modifying CMakeLists files, providing clearer project structure and easier usage.
			# Updated all READMEs that contains step by step instructions for how to build and run the examples.

			=== Problems ===

			# Proper destruction of the msg interface. In RL-TCP example, I had reference counting issue (the reference count didn't go to zero so an object was not destroyed), and fixed reference count by replacing some Ptr<> with raw pointer. There may be other better ways to solve that.
			# Because the msg interface must have only one instance that provides synchronized access of shared memory segment, I use a local variable in a source file so that many functions in different classes can have access to the only interface. I noticed in ns-3 a SingleTon class is provided, is that a better way to define the msg interface?

			=== Todo next week ===

			# Provide some initial benchmark of Gym interface with ns3-gym.
			# Do midterm evaluation.