GSOC2024RLUsability5GFinalReport
Project Overview
- Project Name: Enhancement of RL Approach Accessibility in NR
- Student: Hyerin Kim
- Mentors: Katerina Koutlia, Amir Ashtari, Bijana Bojovic, Gabriel Ferreira
- Project Goals: In this project, I will design a new RL based MAC scheduler of NR and implement it in 5g-lena integrating with ns3-gym. Additionally, I will enhance the usability of 5G-lena in terms of RL approach by providing an example using the designed RL based scheduler.
- Project Proposal: https://summerofcode.withgoogle.com/programs/2024/projects/vPuZgTe1
- Project Wiki: GSOC2024RLUsability5G
Merge Requests and Commits
I maintained a single branch for all work during GSoC: gsoc24-nr-usability
Merge Requests
All the following activities can be easily reviewed on the following MR:
| No. | Name | Status |
|---|---|---|
| [1] | Draft: GSoC2024: RL-based Scheduler | Draft |
Milestones
During the project, I pushed 170+ commits and squashed them to about 70 commits for merge. I managed all commits during the work in my personal repository, named "5g-lena-integrated-with-ns-3-gym/gsoc24-nr-usability" by organizing them into milestones and issues
| No. | Name | Period |
|---|---|---|
| [1] | Design Scenario | Jun 10, 2024–Jun 17, 2024 |
| [2] | Drafting an AI scheduler | Jul 11, 2024–Jul 17, 2024 |
| [3] | Develop an RL-based scheduler | Jul 18, 2024–Jul 24, 2024 |
| [4] | Update the RL-based Scheduler (Code Refactoring) | Jul 24, 2024–Jul 31, 2024 |
| [5] | Create test | Jul 29, 2024–Aug 11, 2024 |
| [6] | Develop Gym Interface in "cttc-nr-rl-based-sched" Example | Aug 12, 2024–Aug 25, 2024 |
| [7] | Develop Gym Python Scripts | Aug 21, 2024–Oct 2, 2024 |
| [8] | Resolve comments in MR: cttc-lena/nr!166 | Sep 2, 2024–Oct 6, 2024 |
Project Details
Phase 1: Design example
During this phase, I became familiar with 5g-lena by studying cttc-nr-demo example and cttc-nr-demo tutorial, and analyzed the existing schedulers in 5g-lena (i.e., QoS, PF, RR) through cttc-nr-simple-qos-sched example, cttc-nr-multi-flow-qos-sched example, and NR module documentation.
Based on this studies, I designed draft scenario which be applied RL-based scheduler.
Activities
- Familiar with 5g-lena (Week 1, Week 2)
- Design Scenario (Week 3) (Milestone 1)
Phase 2: Design RL based Scheduler
During this phrase, I designed RL-based scheduler which can be user-friendly and easy to reuse. Also, set parameters for the number of traffic and traffic types on the example to provide various scenarios to users.
The goal of scheduler is decided based on the problems of the QoS scheduler with QoS LC assignment that the non-GBR flow with DC-GRB flow in a UE have benefit and a single non-GBR flow suffers from starvation due to this benefit. Also, we considered the structure of scheduler is easy to understand.
After setting the goal of RL-base scheduler, I designed how to implement the designed RL-based scheduler in 5g-lena and ns3-gym. To design the RL process, I drew the UML diagram that represents the relationship between scheduler classes and the sequences among methods in the classes. While proceeding this work, I refactored the code based on the review on MR 1.
I developed the process of AI schedulers side of 5G-LENA, that processes data from active UEs and sends it via Notify callback to Ns3-Gym. To verify the code, I also created the unit test which verifies the functionality of the callback used for invoking the ns3-gym module during the resource assigning process.
Activities
- Design Scheduler (Week 4-5)
- Design RL Process (Week 6)
- Implementation of RL-based scheduler in 5g lena (Week 7-9) (Milestone 2, 3)
- Refactor code (Week 9) (Milestone 4)
- Create Unit Test (Week 10-11) (Milestone 5)
Phase 3: RL Integration
During this phase, I integrated the
Activities
- Develop the ns3-gym interface in the RL 5G-LENA example (Week 12) (Milestone 6)
- Develop a python gym script for a simple test (Week 13) (Milestone 7)
- Develop a python gym script for Proximal Policy Optimization (PPO) model (Week 14) (Milestone 7)
Phase 4: Refine Code and Evaluation
Activities
- Address comments in MR 1 (Week 15-16, Week 19) (Milestone 8)
- Conduct Simulation Campaign (Week 17-18)
- Refine Code and RL Models (Week 17-18) (Milestone 7, 8)
- Squash commits (Week 19)