= Project Overview =

* '''Project Name''': [https://summerofcode.withgoogle.com/programs/2023/projects/oeDFknbT IUNS-3 5G NR: Improving the Usability of ns-3's 5G NR Module]
* '''Contributor''': Giovanni Grieco
* '''Mentors''': Tom Henderson, Katerina Koutlia, and Biljana Bojovic
* '''Main Goal''': The primary objective of this project was to enhance the user-friendliness of the 5G NR module within ns-3. This was achieved by providing an inviting user experience and a detailed tutorial that is specifically designed to help users understand the inner workings of a simulated 5G Radio Access Network (RAN) during their initial usage.
* '''Useful links''':
** [https://www.nsnam.org/mediawiki/index.php?title=GSOC20235GUsability Project wiki page] - used to track the work in progress.
** [https://gitlab.com/ggrieco/gsoc23-iuns-3 Project patch set repository] - used to archive all the work done for this GSoC
** [https://gitlab.com/nsnam/ns-3-dev ns-3-dev Official Repository] - will be referenced in the project.
** [https://gitlab.com/cttc-lena/nr/ nr Official Repository] - used to push MRs in output of this GSoC project.

= Milestones =

# Provide a description of how packets flow through the 5G NR RAN. Be able to answer the question "what happens to the traffic between a gNB and a UE?"
# Improve or add documentation when necessary, in order to address usability issues in understanding fundamental example scenarios, such as <code>cttc-nr-demo</code>.
# Enhance packet/frame tracing capabilities by offering flexible log messages and trace sources.
# Review and refactor code to improve clarity on example scenarios.
# Be able to measure latency at different layers of the NR stack.

= Activities =

* ns-3-dev and NR were forked at their respective versions: 3.39 and 2.5.x, creating the following two personal repositories: [https://gitlab.com/ggrieco/ns-3-dev ggrieco/ns-3-dev] and [https://gitlab.com/ggrieco/nr ggrieco/nr].
* Analyzed the <code>cttc-nr-demo</code> example for usability issues.
* Helped mentors in fixing pipeline configuration issues, hindering the correct checking of build and code style.
* Presented work being done and merged over the NR module at [https://www.nsnam.org/research/wns3/wns3-2023/program/ WNS3 2023], gathering interesting feedback from the ns-3 community.
* Created a tutorial to better understand how the NR module works, based on <code>cttc-nr-demo</code> example.
* Various documentation and log improvements tracked by MRs.

== Deviations ==

The project focused on milestones different from the ones listed in the starting project idea after internal meetings with mentors and feedback gathered during the Community Bonding Period.

== Merge Requests ==

All the following activities can be easily reviewed on the following MRs:

{| class="wikitable"
|+ Merge Requests
|-
! No. !! Name !! Status !! Archived Patchset
|-
| 1 || [https://gitlab.com/cttc-lena/nr/-/merge_requests/50 !50 docs: Update sqlite support message] || Merged || [https://gitlab.com/ggrieco/gsoc23-iuns-3/-/tree/main/nr/merged?ref_type=heads link]
|-
| 2 || [https://gitlab.com/cttc-lena/nr/-/merge_requests/51 !51 nr-helper: Drop undefined method] || Merged || [https://gitlab.com/ggrieco/gsoc23-iuns-3/-/tree/main/nr/merged?ref_type=heads link]
|-
| 3 || [https://gitlab.com/cttc-lena/nr/-/merge_requests/52 !52 Example Usability Improvements] || In review || [https://gitlab.com/ggrieco/gsoc23-iuns-3/-/tree/main/nr/MR52?ref_type=heads link]
|-
| 4 || [https://gitlab.com/cttc-lena/nr/-/merge_requests/53 !53 examples: Fix CommandLine declaration for doxygen] || Merged || [https://gitlab.com/ggrieco/gsoc23-iuns-3/-/tree/main/nr/merged?ref_type=heads link]
|-
| 5 || [https://gitlab.com/cttc-lena/nr/-/merge_requests/54 !54 gitlab-ci-gcc: Fix and upgrade build pipeline] || Merged || [https://gitlab.com/ggrieco/gsoc23-iuns-3/-/tree/main/nr/merged?ref_type=heads link]
|-
| 6 || [https://gitlab.com/cttc-lena/nr/-/merge_requests/55 !55 examples, docs: Update TR 38.900 references to TR 38.901] || Refused || [todo link]
|-
| 7 || [https://gitlab.com/cttc-lena/nr/-/merge_requests/56 !56 Readme: Reference the Doxygen tutorial to learn how to run examples] || Merged || [https://gitlab.com/ggrieco/gsoc23-iuns-3/-/tree/main/nr/merged?ref_type=heads link]
|-
| 8 || [https://gitlab.com/cttc-lena/nr/-/merge_requests/57 !57 cttc-nr-demo: Add reference to TS 38.300] || Refused || [todo link]
|-
| 9 || [https://gitlab.com/cttc-lena/nr/-/merge_requests/58 !58 cttc-nr-demo: Add reference to ns-3 logging tutorial] || Refused || [todo link]
|-
| 10 || [https://gitlab.com/cttc-lena/nr/-/merge_requests/59 !59 cttc-nr-demo: Add lower-bound frequency check as per TR 39.901] || Merged || [https://gitlab.com/ggrieco/gsoc23-iuns-3/-/tree/main/nr/merged?ref_type=heads link]
|-
| 11 || [https://gitlab.com/cttc-lena/nr/-/merge_requests/60 !60 nr-radio-environment-map-helper: Remove unused variable] || Refused || [todo link]
|-
| 12 || [https://gitlab.com/cttc-lena/nr/-/merge_requests/61 !61 cttc-nr-demo: Clarify GridScenarioHelper usage] || In review || [https://gitlab.com/ggrieco/gsoc23-iuns-3/-/tree/main/nr/MR61?ref_type=heads link]
|-
| 13 || [https://gitlab.com/cttc-lena/nr/-/merge_requests/62 !62 grid-scenario-helper: Add documentation] || In review || [https://gitlab.com/ggrieco/gsoc23-iuns-3/-/tree/main/nr/MR62?ref_type=heads link]
|-
| 14 || [https://gitlab.com/cttc-lena/nr/-/merge_requests/63 !63 gitlab-ci: Fix SSL errors on git clone] || Merged || [https://gitlab.com/ggrieco/gsoc23-iuns-3/-/tree/main/nr/merged?ref_type=heads link]
|-
| 15 || [https://gitlab.com/cttc-lena/nr/-/merge_requests/65 !65 cttc-nr-demo: Usability improvements over Packet Tracing] || In review || [https://gitlab.com/ggrieco/gsoc23-iuns-3/-/tree/main/nr/MR65?ref_type=heads link]
|-
| 16 || [https://gitlab.com/cttc-lena/nr/-/merge_requests/67 !67 cttc-nr-demo: Add references to standards and ns-3 docs] || In review || [https://gitlab.com/ggrieco/gsoc23-iuns-3/-/tree/main/nr/MR67?ref_type=heads link]
|-
| 17 || [https://gitlab.com/cttc-lena/nr/-/merge_requests/68 !68 logs: Demote Informational logs messages to Function trace messages] || In review || [https://gitlab.com/ggrieco/gsoc23-iuns-3/-/tree/main/nr/MR68?ref_type=heads link]
|-
| 18 || [https://gitlab.com/cttc-lena/nr/-/merge_requests/71 !71 cttc-nr-demo Tutorial] || In review || [https://gitlab.com/ggrieco/gsoc23-iuns-3/-/tree/main/nr/MR71?ref_type=heads link]
|}

= Future Works =

Since the milestones are flexible and subject to continuous enhancement, future efforts can be directed towards:

# Completing the pending MRs that are currently under review.
# Enhancing the tutorial to provide clearer explanations for certain aspects of the simulation, particularly those pertaining to the MAC and PHY layers, in order to align more effectively with the stated milestones.
# Improving the overall state of documentation and code readability of example scenarios.

= Acronyms =

* Centre Tecnològic de Telecomunicacions de Catalunya (CTTC)
* Google Summer of Code (GSoC)
* Medium Access Control (MAC)
* Merge Request (MR)
* New Radio (NR)
* Next Generation Node-B (gNB)
* Radio Access Network (RAN)
* User Equipment (UE)
* Workshop on ns-3 (WNS3)

GSOC20235GUsability

2023-09-09T15:21:39Z

Ggrieco: /* Week 15 [Sep 04 - Sep 09] */

GSOC20235GUsabilityFinalReport

Started the analysis of [https://gitlab.com/cttc-lena/nr/-/blob/master/examples/cttc-nr-demo.cc `cttc-nr-demo`] to probe around usability issues and what I can do to solve them.

=== Initial Doxygen comments - lines: 7-29 ===
This example describes how to setup a simulation using the 3GPP channel model from TR 38.900.

The 3GPP TR 38.900 is part of Release 15 regarding the ''Study on channel model for frequency spectrum above 6 GHz''. It is freely available at the [https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=2991 3GPP Portal] and on [https://portal.etsi.org/webapp/workprogram/Report_WorkItem.asp?WKI_ID=55982 ETSI] as PDF.

First of all, please notice that here we are dealing with a standard that is marked as obsolete, as per comment by John M Meredith make in 2018:
This spec is not maintained after v15.0.0. It is superseded by 38.901.
which is the ''Study on channel model for frequencies from 0.5 to 100 GHz'' and it is available on the [https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3173 3GPP Portal].

So, first of all, are we sure that we are working on code that provides the latest standard implementation Out of the Box? A quick search
$ grep -ri 'TR 38.900' contrib/nr | wc -l
16
$ grep -ri 'TR 38.901' contrib/nr | wc -l
9
reveals that the module has been updated to support the more recent [https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3173 TR 38.901].

{| border=1 cellspacing=0 cellpadding=3
|+ Possible Tasks
! ID !! Task
|-
! T1
| Does `cttc-nr-demo` also follows [https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3173 TR 38.901]?
|-
! T2
| Giving the current state of the code, are we sure to follow 100% TR 38.900, even though there is code that refers to [https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3173 TR 38.901]?
|-
! T3
| Is this code also compliant to [https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3173 TR 38.901]? If so, can we explicitly say it?
|-
|}

This example consists of a simple grid topology, in which you can choose the number of gNbs and UEs. Have a look at the possible parameters to know what you can configure through the command line.

First of all, there is a small typo on "gNb", as it should be written as '''gNB'''. This term is defined in [https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3191 TS 38.300]:
'''gNB''': node providing NR user plane and control plane protocol terminations towards the UE, and connected via the NG interface to the 5GC

This is not the only part where gNB is spelled wrongly:
$ grep -r 'gNb' contrib/nr/ | wc -l
423

Furthermore, here we talk about the "command line". There is no reference of how such command line interface works in the [https://cttc-lena.gitlab.io/nr/nrmodule.pdf NR official manual], neither in the repository [https://gitlab.com/cttc-lena/nr/-/blob/master/README.md README.md file]. Only the [https://cttc-lena.gitlab.io/nr/html/cttc-nr-demo_8cc.html doxygen] documentation reports examples of how its syntax, but such information is not readily available for the novel user.

{| border=1 cellspacing=0 cellpadding=3
|+ Possible Tasks
! ID !! Task
|-
! T4
| Given that we are writing code that follows consolidated standards, shall we adhere (and thus, fix) the spelling of official terms?
|-
! T5
| Shall we explicitly define how the command line interface works to run a NR module example in the project README and/or manual?
|-
|}

With the default configuration, the example will create two flows that will go through two different subband numerologies (or bandwidth parts). For that, specifically, two bands are created, each with a single Carrier Component (CC), and each CC containing one bandwidth part (BWP).

Here we use some technical terms that are related to specific features of NR. This is aggravate by the fact that we are using technical acronyms, i.e., CC, without writing them in extended form. Unfortunately, a quick search of "subband numerologies", "bandwidth part", and "CC" over the aforementioned TR 38.900 returns no explanation or usage.

Consequently, this paragraph is hard to understand if the user does not already have a broader understanding of the several standards that characterize NR, or 3GPP Release 16 (and higher) in general.

{| border=1 cellspacing=0 cellpadding=3
|+ Possible Tasks
! ID !! Task
|-
! T6
| Given that `cttc-nr-demo` should be a welcoming demo: a reading of [https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3191 3GPP TS 38.300] ''"NR; NR and NG-RAN Overall description; Stage-2"'' should be recommended, even before mentioning TR 38.900, as it provides the basic definitions about Carrier Component (CC), sub-band numerologies, and Bandwidth Part (BWP). While CC is mentioned in Section 5.4.1, the remaining twos are defined in Section 5.1.
|-
! T7
| User profile: what is the type of user that we want to address with this example? Does he/she/_ already know all the NR standards? How about ns-3? How about the programming language, i.e., C++? Can he/she/_ may be a student, still learning all of these things, even the basic theory behind mobile communications?
|-
|}

The example will print on-screen the end-to-end result of one (or two) flows, as well as writing them on a file.

This output appears to be too strict. As it is quite succinct for an experienced user, who may easily imagine the node positions, its stack configuration, its traffic, and the overall network topology. On the contrary, an inexperienced user may expect some context to fully understand these Key Performance Indices (KPIs) and statistics.

{| border=1 cellspacing=0 cellpadding=3
|+ Possible Tasks
! ID !! Task
|-
! T8
| By default, `cttc-nr-demo` should output additional contextual information to clarify the statistics that are already shown. This can range from the position of the nodes to their stack configuration, traffic in easily accessible PCAP format, and overall network topology.
|-
|}

With this line, we will be able to see the logs of the file by enabling the component "CttcNrDemo".

Log components are not explained in the [https://cttc-lena.gitlab.io/nr/nrmodule.pdf), neither in the [README](https://gitlab.com/cttc-lena/nr/-/blob/master/README.md NR official manual].

{| border=1 cellspacing=0 cellpadding=3
|+ Possible Tasks
! ID !! Task
|-
! T9
| Provide some references on how the user can work with log components in NR module, or ns-3 in general. This [https://www.nsnam.org/docs/release/3.38/tutorial/html/tweaking.html#using-the-logging-module tutorial] may be helpful.
|-
|}

=== Code lines 148-153 ===
/*
* Check if the frequency is in the allowed range.
* If you need to add other checks, here is the best position to put them.
*/
NS_ABORT_IF(centralFrequencyBand1 > 100e9);
NS_ABORT_IF(centralFrequencyBand2 > 100e9);

According to [https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3173 TR 38.901], we miss the lower bound limit of 0.5 GHz.

{| border=1 cellspacing=0 cellpadding=3
|+ Possible Tasks
! ID !! Task
|-
! T10
| Shall we extend such assertion to include the [https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3173 TR 38.901] check at its lower bound?
|-
|}

=== Code lines 155-170 ===
/*
* If the logging variable is set to true, enable the log of some components
* through the code. The same effect can be obtained through the use
* of the NS_LOG environment variable:
*
* export NS_LOG="UdpClient=level_info|prefix_time|prefix_func|prefix_node:UdpServer=..."
*
* Usually, the environment variable way is preferred, as it is more customizable,
* and more expressive.
*/
if (logging)
{
LogComponentEnable("UdpClient", LOG_LEVEL_INFO);
LogComponentEnable("UdpServer", LOG_LEVEL_INFO);
LogComponentEnable("LtePdcp", LOG_LEVEL_INFO);
}

This information would solve `T9` by anticipating such information.

{| border=1 cellspacing=0 cellpadding=3
|+ Possible Tasks
! ID !! Task
|-
! T11
| Shall we move this explanation to line ~58? How about repeating it in the [https://cttc-lena.gitlab.io/nr/nrmodule.pdf NR official manual] and/or in the [https://gitlab.com/cttc-lena/nr/-/blob/master/README.md README.md file]?
|-
|}

=== Code lines 172-176 ===
/*
* Default values for the simulation. We are progressively removing all
* the instances of SetDefault, but we need it for legacy code (LTE)
*/
Config::SetDefault("ns3::LteRlcUm::MaxTxBufferSize", UintegerValue(999999999));

This does not seem to be entirely related to the example per se, given that it misses key information on why such scenario should have such value to be set. Furthermore, such value is applied across all example scenarios. To this end, it would be better to hide such code from the user eyes, in order to not cause confusion.

{| border=1 cellspacing=0 cellpadding=3
|+ Possible Tasks
! ID !! Task
|-
! T12
| What is the role of this line? Shall we move this code to `ns3::NrHelper`? Maybe to a method called `void SetupDefaultNrNsaParameters()`?
|-
|}

=== Code lines 178-198 ===
/*
* Create the scenario. In our examples, we heavily use helpers that setup
* the gnbs and ue following a pre-defined pattern. Please have a look at the
* GridScenarioHelper documentation to see how the nodes will be distributed.
*/
int64_t randomStream = 1;
GridScenarioHelper gridScenario;
gridScenario.SetRows(1);
gridScenario.SetColumns(gNbNum);
gridScenario.SetHorizontalBsDistance(5.0);
gridScenario.SetVerticalBsDistance(5.0);
gridScenario.SetBsHeight(1.5);
gridScenario.SetUtHeight(1.5);
// must be set before BS number
gridScenario.SetSectorization(GridScenarioHelper::SINGLE);
gridScenario.SetBsNumber(gNbNum);
gridScenario.SetUtNumber(ueNumPergNb * gNbNum);
gridScenario.SetScenarioHeight(3); // Create a 3x3 scenario where the UE will
gridScenario.SetScenarioLength(3); // be distribuited.
randomStream += gridScenario.AssignStreams(randomStream);
gridScenario.CreateScenario();

'''`ns3::GridScenarioHelper` documentation is marked as TODO''' in the [https://gitlab.com/cttc-lena/nr/-/blob/master/helper/grid-scenario-helper.h source file].
Furthermore, ''distance'', ''length'', and ''height'' properties do not provide any unit of measurement. This is aggravated by the lack of any sort of unit to be used in both code (we have a generic `double`) and documentation. Intuitively, the user, which may come from any place of Earth, can be confused on using the [https://www.nist.gov/pml/owm/metric-si/si-units International System of Units (SI)], the metric system, or the imperial one.

{| border=1 cellspacing=0 cellpadding=3
|+ Possible Tasks
! ID !! Task
|-
! T13
| Shall we document `ns3::GridScenarioHelper`?
|-
! T14
| Is it ok to provide the information over the unit of measurements to be used? Where should we say so? Is it ok to provide macros/helpers to be able to specify measurements in metric and/or imperial? Shall we stick to the SI?
|-
|}

GSOC20235GUsability

2023-06-12T13:04:13Z

2023-05-26T15:30:07Z

Ggrieco: