3.4.2. Data Collection Helpers

The full flexibility of the data collection framework is provided by the interconnection of probes, collectors, and aggregators. Performing all of these interconnections leads to many configuration statements in user programs. For ease of use, some of the most common operations can be combined and encapsulated in helper functions. In addition, some statements involving ns-3 trace sources do not have Python bindings, due to limitations in the bindings.

3.4.2.1. Data Collection Helpers Overview

In this section, we provide an overview of some helper classes that have been created to ease the configuration of the data collection framework for some common use cases. The helpers allow users to form common operations with only a few statements in their C++ or Python programs. But, this ease of use comes at the cost of significantly less flexibility than low-level configuration can provide, and the need to explicitly code support for new Probe types into the helpers (to work around an issue described below).

The emphasis on the current helpers is to marshal data out of ns-3 trace sources into gnuplot plots or text files, without a high degree of output customization or statistical processing (initially). Also, the use is constrained to the available probe types in ns-3. Later sections of this documentation will go into more detail about creating new Probe types, as well as details about hooking together Probes, Collectors, and Aggregators in custom arrangements.

To date, two Data Collection helpers have been implemented:

  • GnuplotHelper
  • FileHelper

3.4.2.2. GnuplotHelper

The GnuplotHelper is a helper class for producing output files used to make gnuplots. The overall goal is to provide the ability for users to quickly make plots from data exported in ns-3 trace sources. By default, a minimal amount of data transformation is performed; the objective is to generate plots with as few (default) configuration statements as possible.

3.4.2.2.1. GnuplotHelper Overview

The GnuplotHelper will create 3 different files at the end of the simulation:

  • A space separated gnuplot data file
  • A gnuplot control file
  • A shell script to generate the gnuplot

There are two configuration statements that are needed to produce plots. The first statement configures the plot (filename, title, legends, and output type, where the output type defaults to PNG if unspecified):

void ConfigurePlot(const std::string &outputFileNameWithoutExtension,
                   const std::string &title,
                   const std::string &xLegend,
                   const std::string &yLegend,
                   const std::string &terminalType = ".png");

The second statement hooks the trace source of interest:

void PlotProbe(const std::string &typeId,
               const std::string &path,
               const std::string &probeTraceSource,
               const std::string &title);

The arguments are as follows:

  • typeId: The ns-3 TypeId of the Probe
  • path: The path in the ns-3 configuration namespace to one or more trace sources
  • probeTraceSource: Which output of the probe (itself a trace source) should be plotted
  • title: The title to associate with the dataset(s) (in the gnuplot legend)

A variant on the PlotProbe above is to specify a fifth optional argument that controls where in the plot the key (legend) is placed.

A fully worked example (from seventh.cc) is shown below:

// Create the gnuplot helper.
GnuplotHelper plotHelper;

// Configure the plot.
// Configure the plot.  The first argument is the file name prefix
// for the output files generated.  The second, third, and fourth
// arguments are, respectively, the plot title, x-axis, and y-axis labels
plotHelper.ConfigurePlot("seventh-packet-byte-count",
                         "Packet Byte Count vs. Time",
                         "Time(Seconds)",
                         "Packet Byte Count",
                         "png");

// Specify the probe type, trace source path (in configuration namespace), and
// probe output trace source ("OutputBytes") to plot.  The fourth argument
// specifies the name of the data series label on the plot.  The last
// argument formats the plot by specifying where the key should be placed.
plotHelper.PlotProbe(probeType,
                     tracePath,
                     "OutputBytes",
                     "Packet Byte Count",
                     GnuplotAggregator::KEY_BELOW);

In this example, the probeType and tracePath are as follows (for IPv4):

probeType = "ns3::Ipv4PacketProbe";
tracePath = "/NodeList/*/$ns3::Ipv4L3Protocol/Tx";

The probeType is a key parameter for this helper to work. This TypeId must be registered in the system, and the signature on the Probe’s trace sink must match that of the trace source it is being hooked to. Probe types are pre-defined for a number of data types corresponding to ns-3 traced values, and for a few other trace source signatures such as the ‘Tx’ trace source of ns3::Ipv4L3Protocol class.

Note that the trace source path specified may contain wildcards. In this case, multiple datasets are plotted on one plot; one for each matched path.

The main output produced will be three files:

seventh-packet-byte-count.dat
seventh-packet-byte-count.plt
seventh-packet-byte-count.sh

At this point, users can either hand edit the .plt file for further customizations, or just run it through gnuplot. Running sh seventh-packet-byte-count.sh simply runs the plot through gnuplot, as shown below.

_images/seventh-packet-byte-count.png

2-D Gnuplot Created by seventh.cc Example.

It can be seen that the key elements (legend, title, legend placement, xlabel, ylabel, and path for the data) are all placed on the plot. Since there were two matches to the configuration path provided, the two data series are shown:

  • Packet Byte Count-0 corresponds to /NodeList/0/$ns3::Ipv4L3Protocol/Tx
  • Packet Byte Count-1 corresponds to /NodeList/1/$ns3::Ipv4L3Protocol/Tx

3.4.2.2.2. GnuplotHelper ConfigurePlot

The GnuplotHelper’s ConfigurePlot() function can be used to configure plots.

It has the following prototype:

void ConfigurePlot(const std::string &outputFileNameWithoutExtension,
                   const std::string &title,
                   const std::string &xLegend,
                   const std::string &yLegend,
                   const std::string &terminalType = ".png");

It has the following arguments:

Argument Description
outputFileNameWithoutExtension Name of gnuplot related files to write with no extension.
title Plot title string to use for this plot.
xLegend The legend for the x horizontal axis.
yLegend The legend for the y vertical axis.
terminalType Terminal type setting string for output. The default terminal type is “png”.

The GnuplotHelper’s ConfigurePlot() function configures plot related parameters for this gnuplot helper so that it will create a space separated gnuplot data file named outputFileNameWithoutExtension + “.dat”, a gnuplot control file named outputFileNameWithoutExtension + “.plt”, and a shell script to generate the gnuplot named outputFileNameWithoutExtension + “.sh”.

An example of how to use this function can be seen in the seventh.cc code described above where it was used as follows:

plotHelper.ConfigurePlot("seventh-packet-byte-count",
                         "Packet Byte Count vs. Time",
                         "Time(Seconds)",
                         "Packet Byte Count",
                         "png");

3.4.2.2.3. GnuplotHelper PlotProbe

The GnuplotHelper’s PlotProbe() function can be used to plot values generated by probes.

It has the following prototype:

void PlotProbe(const std::string &typeId,
               const std::string &path,
               const std::string &probeTraceSource,
               const std::string &title,
               enum GnuplotAggregator::KeyLocation keyLocation = GnuplotAggregator::KEY_INSIDE);

It has the following arguments:

Argument Description
typeId The type ID for the probe created by this helper.
path Config path to access the trace source.
probeTraceSource The probe trace source to access.
title The title to be associated to this dataset
keyLocation The location of the key in the plot. The default location is inside.

The GnuplotHelper’s PlotProbe() function plots a dataset generated by hooking the ns-3 trace source with a probe created by the helper, and then plotting the values from the probeTraceSource. The dataset will have the provided title, and will consist of the ‘newValue’ at each timestamp.

If the config path has more than one match in the system because there is a wildcard, then one dataset for each match will be plotted. The dataset titles will be suffixed with the matched characters for each of the wildcards in the config path, separated by spaces. For example, if the proposed dataset title is the string “bytes”, and there are two wildcards in the path, then dataset titles like “bytes-0 0” or “bytes-12 9” will be possible as labels for the datasets that are plotted.

An example of how to use this function can be seen in the seventh.cc code described above where it was used (with variable substitution) as follows:

plotHelper.PlotProbe("ns3::Ipv4PacketProbe",
                     "/NodeList/*/$ns3::Ipv4L3Protocol/Tx",
                     "OutputBytes",
                     "Packet Byte Count",
                     GnuplotAggregator::KEY_BELOW);

3.4.2.2.4. Other Examples

3.4.2.2.4.1. Gnuplot Helper Example

A slightly simpler example than the seventh.cc example can be found in src/stats/examples/gnuplot-helper-example.cc. The following 2-D gnuplot was created using the example.

_images/gnuplot-helper-example.png

2-D Gnuplot Created by gnuplot-helper-example.cc Example.

In this example, there is an Emitter object that increments its counter according to a Poisson process and then emits the counter’s value as a trace source.

Ptr<Emitter> emitter = CreateObject<Emitter>();
Names::Add("/Names/Emitter", emitter);

Note that because there are no wildcards in the path used below, only 1 datastream was drawn in the plot. This single datastream in the plot is simply labeled “Emitter Count”, with no extra suffixes like one would see if there were wildcards in the path.

// Create the gnuplot helper.
GnuplotHelper plotHelper;

// Configure the plot.
plotHelper.ConfigurePlot("gnuplot-helper-example",
                         "Emitter Counts vs. Time",
                         "Time(Seconds)",
                         "Emitter Count",
                         "png");

// Plot the values generated by the probe.  The path that we provide
// helps to disambiguate the source of the trace.
plotHelper.PlotProbe("ns3::Uinteger32Probe",
                     "/Names/Emitter/Counter",
                     "Output",
                     "Emitter Count",
                     GnuplotAggregator::KEY_INSIDE);

3.4.2.3. FileHelper

The FileHelper is a helper class used to put data values into a file. The overall goal is to provide the ability for users to quickly make formatted text files from data exported in ns-3 trace sources. By default, a minimal amount of data transformation is performed; the objective is to generate files with as few (default) configuration statements as possible.

3.4.2.3.1. FileHelper Overview

The FileHelper will create 1 or more text files at the end of the simulation.

The FileHelper can create 4 different types of text files:

  • Formatted
  • Space separated (the default)
  • Comma separated
  • Tab separated

Formatted files use C-style format strings and the sprintf() function to print their values in the file being written.

The following text file with 2 columns of formatted values named seventh-packet-byte-count-0.txt was created using more new code that was added to the original ns-3 Tutorial example’s code. Only the first 10 lines of this file are shown here for brevity.

Time (Seconds) = 1.000e+00    Packet Byte Count = 40
Time (Seconds) = 1.004e+00    Packet Byte Count = 40
Time (Seconds) = 1.004e+00    Packet Byte Count = 576
Time (Seconds) = 1.009e+00    Packet Byte Count = 576
Time (Seconds) = 1.009e+00    Packet Byte Count = 576
Time (Seconds) = 1.015e+00    Packet Byte Count = 512
Time (Seconds) = 1.017e+00    Packet Byte Count = 576
Time (Seconds) = 1.017e+00    Packet Byte Count = 544
Time (Seconds) = 1.025e+00    Packet Byte Count = 576
Time (Seconds) = 1.025e+00    Packet Byte Count = 544

...

The following different text file with 2 columns of formatted values named seventh-packet-byte-count-1.txt was also created using the same new code that was added to the original ns-3 Tutorial example’s code. Only the first 10 lines of this file are shown here for brevity.

Time (Seconds) = 1.002e+00    Packet Byte Count = 40
Time (Seconds) = 1.007e+00    Packet Byte Count = 40
Time (Seconds) = 1.013e+00    Packet Byte Count = 40
Time (Seconds) = 1.020e+00    Packet Byte Count = 40
Time (Seconds) = 1.028e+00    Packet Byte Count = 40
Time (Seconds) = 1.036e+00    Packet Byte Count = 40
Time (Seconds) = 1.045e+00    Packet Byte Count = 40
Time (Seconds) = 1.053e+00    Packet Byte Count = 40
Time (Seconds) = 1.061e+00    Packet Byte Count = 40
Time (Seconds) = 1.069e+00    Packet Byte Count = 40

...

The new code that was added to produce the two text files is below. More details about this API will be covered in a later section.

Note that because there were 2 matches for the wildcard in the path, 2 separate text files were created. The first text file, which is named “seventh-packet-byte-count-0.txt”, corresponds to the wildcard match with the “*” replaced with “0”. The second text file, which is named “seventh-packet-byte-count-1.txt”, corresponds to the wildcard match with the “*” replaced with “1”. Also, note that the function call to WriteProbe() will give an error message if there are no matches for a path that contains wildcards.

// Create the file helper.
FileHelper fileHelper;

// Configure the file to be written.
fileHelper.ConfigureFile("seventh-packet-byte-count",
                         FileAggregator::FORMATTED);

// Set the labels for this formatted output file.
fileHelper.Set2dFormat("Time (Seconds) = %.3e\tPacket Byte Count = %.0f");

// Write the values generated by the probe.
fileHelper.WriteProbe("ns3::Ipv4PacketProbe",
                      "/NodeList/*/$ns3::Ipv4L3Protocol/Tx",
                      "OutputBytes");

3.4.2.3.2. FileHelper ConfigureFile

The FileHelper’s ConfigureFile() function can be used to configure text files.

It has the following prototype:

void ConfigureFile(const std::string &outputFileNameWithoutExtension,
                   enum FileAggregator::FileType fileType = FileAggregator::SPACE_SEPARATED);

It has the following arguments:

Argument Description
outputFileNameWithoutExtension Name of output file to write with no extension.
fileType Type of file to write. The default type of file is space separated.

The FileHelper’s ConfigureFile() function configures text file related parameters for the file helper so that it will create a file named outputFileNameWithoutExtension plus possible extra information from wildcard matches plus “.txt” with values printed as specified by fileType. The default file type is space-separated.

An example of how to use this function can be seen in the seventh.cc code described above where it was used as follows:

fileHelper.ConfigureFile("seventh-packet-byte-count",
                         FileAggregator::FORMATTED);

3.4.2.3.3. FileHelper WriteProbe

The FileHelper’s WriteProbe() function can be used to write values generated by probes to text files.

It has the following prototype:

void WriteProbe(const std::string &typeId,
                const std::string &path,
                const std::string &probeTraceSource);

It has the following arguments:

Argument Description
typeId The type ID for the probe to be created.
path Config path to access the trace source.
probeTraceSource The probe trace source to access.

The FileHelper’s WriteProbe() function creates output text files generated by hooking the ns-3 trace source with a probe created by the helper, and then writing the values from the probeTraceSource. The output file names will have the text stored in the member variable m_outputFileNameWithoutExtension plus “.txt”, and will consist of the ‘newValue’ at each timestamp.

If the config path has more than one match in the system because there is a wildcard, then one output file for each match will be created. The output file names will contain the text in m_outputFileNameWithoutExtension plus the matched characters for each of the wildcards in the config path, separated by dashes, plus “.txt”. For example, if the value in m_outputFileNameWithoutExtension is the string “packet-byte-count”, and there are two wildcards in the path, then output file names like “packet-byte-count-0-0.txt” or “packet-byte-count-12-9.txt” will be possible as names for the files that will be created.

An example of how to use this function can be seen in the seventh.cc code described above where it was used as follows:

fileHelper.WriteProbe("ns3::Ipv4PacketProbe",
                      "/NodeList/*/$ns3::Ipv4L3Protocol/Tx",
                      "OutputBytes");

3.4.2.3.4. Other Examples

3.4.2.3.4.1. File Helper Example

A slightly simpler example than the seventh.cc example can be found in src/stats/examples/file-helper-example.cc. This example only uses the FileHelper.

The following text file with 2 columns of formatted values named file-helper-example.txt was created using the example. Only the first 10 lines of this file are shown here for brevity.

Time (Seconds) = 0.203  Count = 1
Time (Seconds) = 0.702  Count = 2
Time (Seconds) = 1.404  Count = 3
Time (Seconds) = 2.368  Count = 4
Time (Seconds) = 3.364  Count = 5
Time (Seconds) = 3.579  Count = 6
Time (Seconds) = 5.873  Count = 7
Time (Seconds) = 6.410  Count = 8
Time (Seconds) = 6.472  Count = 9
...

In this example, there is an Emitter object that increments its counter according to a Poisson process and then emits the counter’s value as a trace source.

Ptr<Emitter> emitter = CreateObject<Emitter>();
Names::Add("/Names/Emitter", emitter);

Note that because there are no wildcards in the path used below, only 1 text file was created. This single text file is simply named “file-helper-example.txt”, with no extra suffixes like you would see if there were wildcards in the path.

// Create the file helper.
FileHelper fileHelper;

// Configure the file to be written.
fileHelper.ConfigureFile("file-helper-example",
                         FileAggregator::FORMATTED);

// Set the labels for this formatted output file.
fileHelper.Set2dFormat("Time (Seconds) = %.3e\tCount = %.0f");

// Write the values generated by the probe.  The path that we
// provide helps to disambiguate the source of the trace.
fileHelper.WriteProbe("ns3::Uinteger32Probe",
                      "/Names/Emitter/Counter",
                      "Output");

3.4.2.4. Scope and Limitations

Currently, only these Probes have been implemented and connected to the GnuplotHelper and to the FileHelper:

  • BooleanProbe
  • DoubleProbe
  • Uinteger8Probe
  • Uinteger16Probe
  • Uinteger32Probe
  • TimeProbe
  • PacketProbe
  • ApplicationPacketProbe
  • Ipv4PacketProbe

These Probes, therefore, are the only TypeIds available to be used in PlotProbe() and WriteProbe().

In the next few sections, we cover each of the fundamental object types (Probe, Collector, and Aggregator) in more detail, and show how they can be connected together using lower-level API.