View source for GSOC20235GUsability/Analysis:cttc-nr-demo

Started the analysis of [https://gitlab.com/cttc-lena/nr/-/blob/master/examples/cttc-nr-demo.cc `cttc-nr-demo`] to probe around usability issues and what I can do to solve them.

=== Initial Doxygen comments - lines: 7-29 ===
  This example describes how to setup a simulation using the 3GPP channel model from TR 38.900.

The 3GPP TR 38.900 is part of Release 15 regarding the ''Study on channel model for frequency spectrum above 6 GHz''. It is freely available at the [https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=2991 3GPP Portal] and on [https://portal.etsi.org/webapp/workprogram/Report_WorkItem.asp?WKI_ID=55982 ETSI] as PDF.

First of all, please notice that here we are dealing with a standard that is marked as obsolete, as per comment by John M Meredith make in 2018:
  This spec is not maintained after v15.0.0. It is superseded by 38.901.
which is the ''Study on channel model for frequencies from 0.5 to 100 GHz'' and it is available on the [https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3173 3GPP Portal].

So, first of all, are we sure that we are working on code that provides the latest standard implementation Out of the Box? A quick search
  $ grep -ri 'TR 38.900' contrib/nr | wc -l
  16
  $ grep -ri 'TR 38.901' contrib/nr | wc -l
  9
reveals that the module has been updated to support the more recent [https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3173 TR 38.901].

{| border=1 cellspacing=0 cellpadding=3
|+ Possible Tasks
! ID !! Task
|-
! T1
| Does `cttc-nr-demo` also follows [https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3173 TR 38.901]?
|-
! T2
| Giving the current state of the code, are we sure to follow 100% TR 38.900, even though there is code that refers to [https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3173 TR 38.901]?
|-
! T3
| Is this code also compliant to [https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3173 TR 38.901]? If so, can we explicitly say it?
|-
|}

  This example consists of a simple grid topology, in which you can choose the number of gNbs and UEs. Have a look at the possible parameters to know what you can configure through the command line.

First of all, there is a small typo on "gNb", as it should be written as '''gNB'''. This term is defined in [https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3191 TS 38.300]:
 '''gNB''': node providing NR user plane and control plane protocol terminations towards the UE, and connected via the NG interface to the 5GC

This is not the only part where gNB is spelled wrongly:
  $ grep -r 'gNb' contrib/nr/ | wc -l
  423

Furthermore, here we talk about the "command line". There is no reference of how such command line interface works in the [https://cttc-lena.gitlab.io/nr/nrmodule.pdf NR official manual], neither in the repository [https://gitlab.com/cttc-lena/nr/-/blob/master/README.md README.md file]. Only the [https://cttc-lena.gitlab.io/nr/html/cttc-nr-demo_8cc.html doxygen] documentation reports examples of how its syntax, but such information is not readily available for the novel user.

{| border=1 cellspacing=0 cellpadding=3
|+ Possible Tasks
! ID !! Task
|-
! T4
| Given that we are writing code that follows consolidated standards, shall we adhere (and thus, fix) the spelling of official terms?
|-
! T5
| Shall we explicitly define how the command line interface works to run a NR module example in the project README and/or manual?
|-
|}

With the default configuration, the example will create two flows that will go through two different subband numerologies (or bandwidth parts). For that, specifically, two bands are created, each with a single Carrier Component (CC), and each CC containing one bandwidth part (BWP).

Here we use some technical terms that are related to specific features of NR. This is aggravate by the fact that we are using technical acronyms, i.e., CC, without writing them in extended form. Unfortunately, a quick search of "subband numerologies", "bandwidth part", and "CC" over the aforementioned TR 38.900 returns no explanation or usage.

Consequently, this paragraph is hard to understand if the user does not already have a broader understanding of the several standards that characterize NR, or 3GPP Release 16 (and higher) in general.

{| border=1 cellspacing=0 cellpadding=3
|+ Possible Tasks
! ID !! Task
|-
! T6
| Given that `cttc-nr-demo` should be a welcoming demo: a reading of [https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3191 3GPP TS 38.300] ''"NR; NR and NG-RAN Overall description; Stage-2"'' should be recommended, even before mentioning TR 38.900, as it provides the basic definitions about Carrier Component (CC), sub-band numerologies, and Bandwidth Part (BWP). While CC is mentioned in Section 5.4.1, the remaining twos are defined in Section 5.1.
|-
! T7
| User profile: what is the type of user that we want to address with this example? Does he/she/_ already know all the NR standards? How about ns-3? How about the programming language, i.e., C++? Can he/she/_ may be a student, still learning all of these things, even the basic theory behind mobile communications?
|-
|}

  The example will print on-screen the end-to-end result of one (or two) flows, as well as writing them on a file.

This output appears to be too strict. As it is quite succinct for an experienced user, who may easily imagine the node positions, its stack configuration, its traffic, and the overall network topology. On the contrary, an inexperienced user may expect some context to fully understand these Key Performance Indices (KPIs) and statistics.

{| border=1 cellspacing=0 cellpadding=3
|+ Possible Tasks
! ID !! Task
|-
! T8
| By default, `cttc-nr-demo` should output additional contextual information to clarify the statistics that are already shown. This can range from the position of the nodes to their stack configuration, traffic in easily accessible PCAP format, and overall network topology.
|-
|}

  With this line, we will be able to see the logs of the file by enabling the component "CttcNrDemo".

Log components are not explained in the [https://cttc-lena.gitlab.io/nr/nrmodule.pdf), neither in the [README](https://gitlab.com/cttc-lena/nr/-/blob/master/README.md NR official manual].

{| border=1 cellspacing=0 cellpadding=3
|+ Possible Tasks
! ID !! Task
|-
! T9
| Provide some references on how the user can work with log components in NR module, or ns-3 in general. This [https://www.nsnam.org/docs/release/3.38/tutorial/html/tweaking.html#using-the-logging-module tutorial] may be helpful.
|-
|}

=== Code lines 148-153 ===
  /*
   * Check if the frequency is in the allowed range.
   * If you need to add other checks, here is the best position to put them.
   */
  NS_ABORT_IF(centralFrequencyBand1 > 100e9);
  NS_ABORT_IF(centralFrequencyBand2 > 100e9);

According to [https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3173 TR 38.901], we miss the lower bound limit of 0.5 GHz.

{| border=1 cellspacing=0 cellpadding=3
|+ Possible Tasks
! ID !! Task
|-
! T10
| Shall we extend such assertion to include the [https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3173 TR 38.901] check at its lower bound?
|-
|}

=== Code lines 155-170 ===
  /*
   * If the logging variable is set to true, enable the log of some components
   * through the code. The same effect can be obtained through the use
   * of the NS_LOG environment variable:
   *
   * export NS_LOG="UdpClient=level_info|prefix_time|prefix_func|prefix_node:UdpServer=..."
   *
   * Usually, the environment variable way is preferred, as it is more customizable,
   * and more expressive.
   */
  if (logging)
  {
    LogComponentEnable("UdpClient", LOG_LEVEL_INFO);
    LogComponentEnable("UdpServer", LOG_LEVEL_INFO);
    LogComponentEnable("LtePdcp", LOG_LEVEL_INFO);
  }

This information would solve `T9` by anticipating such information.

{| border=1 cellspacing=0 cellpadding=3
|+ Possible Tasks
! ID !! Task
|-
! T11
| Shall we move this explanation to line ~58? How about repeating it in the [https://cttc-lena.gitlab.io/nr/nrmodule.pdf NR official manual] and/or in the [https://gitlab.com/cttc-lena/nr/-/blob/master/README.md README.md file]?
|-
|}

=== Code lines 172-176 ===
  /*
   * Default values for the simulation. We are progressively removing all
   * the instances of SetDefault, but we need it for legacy code (LTE)
   */
  Config::SetDefault("ns3::LteRlcUm::MaxTxBufferSize", UintegerValue(999999999));

This does not seem to be entirely related to the example per se, given that it misses key information on why such scenario should have such value to be set. Furthermore, such value is applied across all example scenarios. To this end, it would be better to hide such code from the user eyes, in order to not cause confusion.

{| border=1 cellspacing=0 cellpadding=3
|+ Possible Tasks
! ID !! Task
|-
! T12
| What is the role of this line? Shall we move this code to `ns3::NrHelper`? Maybe to a method called `void SetupDefaultNrNsaParameters()`?
|-
|}

=== Code lines 178-198 ===
  /*
   * Create the scenario. In our examples, we heavily use helpers that setup
   * the gnbs and ue following a pre-defined pattern. Please have a look at the
   * GridScenarioHelper documentation to see how the nodes will be distributed.
   */
  int64_t randomStream = 1;
  GridScenarioHelper gridScenario;
  gridScenario.SetRows(1);
  gridScenario.SetColumns(gNbNum);
  gridScenario.SetHorizontalBsDistance(5.0);
  gridScenario.SetVerticalBsDistance(5.0);
  gridScenario.SetBsHeight(1.5);
  gridScenario.SetUtHeight(1.5);
  // must be set before BS number
  gridScenario.SetSectorization(GridScenarioHelper::SINGLE);
  gridScenario.SetBsNumber(gNbNum);
  gridScenario.SetUtNumber(ueNumPergNb * gNbNum);
  gridScenario.SetScenarioHeight(3); // Create a 3x3 scenario where the UE will
  gridScenario.SetScenarioLength(3); // be distribuited.
  randomStream += gridScenario.AssignStreams(randomStream);
  gridScenario.CreateScenario();

'''`ns3::GridScenarioHelper` documentation is marked as TODO''' in the [https://gitlab.com/cttc-lena/nr/-/blob/master/helper/grid-scenario-helper.h source file].
Furthermore, ''distance'', ''length'', and ''height'' properties do not provide any unit of measurement. This is aggravated by the lack of any sort of unit to be used in both code (we have a generic `double`) and documentation. Intuitively, the user, which may come from any place of Earth, can be confused on using the [https://www.nist.gov/pml/owm/metric-si/si-units International System of Units (SI)], the metric system, or the imperial one.

{| border=1 cellspacing=0 cellpadding=3
|+ Possible Tasks
! ID !! Task
|-
! T13
| Shall we document `ns3::GridScenarioHelper`?
|-
! T14
| Is it ok to provide the information over the unit of measurements to be used? Where should we say so? Is it ok to provide macros/helpers to be able to specify measurements in metric and/or imperial? Shall we stick to the SI?
|-
|}