GSOC2024RLUsability5G

Main Page - Roadmap - Summer Projects - Project Ideas - Developer FAQ - Tools - Related Projects

HOWTOs - Installation - Troubleshooting - User FAQ - Samples - Models - Education - Contributed Code - Papers

Back to GSoC 2024 projects

Project Overview

• Project Name: Enhancement of RL Approach Accessibility in NR
• Student: Hyerin Kim
• Mentors: Amir Ashtari, Katerina Koutlia, Bijana Bojovic, Gabriel Ferreira
• Google page: https://summerofcode.withgoogle.com/programs/2024/projects/vPuZgTe1
• Project Goals: In this project, I will improve the interface between OpenGymEnv in ns3 and ns3env in the Python ns3 gym module for Multi-Agent Reinforcement Learning (MARL). Currently, ns3gym utilizes the REQ-REP pattern in ZeroMQ as the interface. By incorporating techniques such as client identity and parallel processing of workers in ZMQ, I aim to develop a MARL interface in ns3gym. Additionally, I will enhance the usability of 5G by implementing MARL approaches in 5G examples, with a specific focus on 5G LENA.
• Repository: https://github.com/mye280c37/GSoC-2024
• About Me: I am currently pursuing a Master's degree in Computer Science and Engineering at Seoul National University, Korea. My research at the Mobile Computing & Communications Laboratory focuses on resource allocation methods in NR V2X Sidelink. As an undergraduate, I conducted research on improving spatial reuse in dense Wi-Fi environments and implemented a Reinforcement Learning (RL)-based modified OBSS/PD algorithm using ns3 gym. I believe that participating in GSoC 2024 presents an excellent opportunity for me to contribute to enhancing the usability of 5G and RL experiences on ns-3, while also deepening my understanding of 5G technology, mechanisms, and system architecture.

Milestones

Phase1. Design example (3 weeks)

• Familiar with 5g-lena (2 weeks)
• Design Scenario (e.g., UEs deployment, UEs speed, cell configuration, …) (1 week)

- Define Assumption (e.g., delay, TDMA/OFDMA, …)

Phase2. Design RL based Scheduler (6 +1 weeks)

• Design scheduler (2 weeks)

- input/output

- goal of optimization

• Design RL process (1 week)

- Define suitable RL techniques considering optimization objective of the scheduler and computational complexity

• Implementation of RL based scheduler in 5g lena (3 +1 weeks)

- Create the test

- Create documentation

- Create MR to 5g lena (Milestone 1)

Phase3. RL Integration (4 weeks)

• Design RL framework (1 week)

- Define RL technique

• Develop gym scripts (3 weeks)

- Develop gym python scripts

- Develop ns3 gym interface in RL 5g lena example

- Validate RL process of the example

- Create MR to 5g lena (Milestone 2)

Phase4. Test and Evaluation (3 +1 weeks)

• Evaluate the result of example compared with other schedulers

- Write simulation campaign scripts

- Execute scripts

- Plotting python scripts

• Address review comment of the MR 1 and 2
• Update MR 1 and 2 with necessary modification
• Create brief description of the work and the results for 5g lena blog (Milestone 3)