My previous attempt was not well-received during GSoC, so I thrashed it away. Here I want to come to a general agreement over a possible API architecture.

Features (in my mind) of this  trace helper from the user point of view:

*) Possibility to selectively choose TCP parameters for each socket (congestion control, slow start thresh, etc.)

*) Possibility to easily setup comparisons with different parameters, even in the same simulation (e.g. NewReno vs Cubic)

*) Possibility to selectively choose trace sources to attach to

*) Possibility to generate data sets from this data (e.g. cwnd, ssth, seq number vs time, etc).

Main points to consider in the implementation:

*) Right now all the applications create the socket inside their code, using an attribute to determine the transport type (i.e. UDP or TCP).

*) The sockets are created in the Start () method, so they don't exist before the start of the application

*) Saving all the data in RAM could be heavy; we need some class that offload to disk the data after some time

My proposals:

*) Give a method to ALL applications for set an external-created socket. The helper will take take of creating sockets, installing it into applications (and attaching to trace sources). Example:

TcpTraceHelper tcpHelper;
...
tcpHelper.CreateSocketFor (appContainer); /* for each app in apps; do Ptr<Socket> *s = CreateSocket; s->Connect... ; app->SetSocket (s); done */
...

*) Define a easy and simple language for allowing users to select 0 or more trace source to attach, with a way to identify flows.

Any help or hint is appreciated.

I'm (In reply to natale.patriciello from comment #1)
> My previous attempt was not well-received during GSoC, so I thrashed it
> away. Here I want to come to a general agreement over a possible API
> architecture.
> 
> Features (in my mind) of this  trace helper from the user point of view:
> 
> *) Possibility to selectively choose TCP parameters for each socket
> (congestion control, slow start thresh, etc.)
> 
> *) Possibility to easily setup comparisons with different parameters, even
> in the same simulation (e.g. NewReno vs Cubic)
> 
> *) Possibility to selectively choose trace sources to attach to
> 
> *) Possibility to generate data sets from this data (e.g. cwnd, ssth, seq
> number vs time, etc).

I think we need still to define what output we want and whether we want to do this all within the context of a single program execution, or multiple executions, and work backwards.

it seems to me that if tcp-variants-comparison behavior (i.e. ability to run the same scenario repeatedly, with different TCP variants) is to be generalized, one way is to do it outside of ns-3; that is, have an external script runner and plotter that runs the program once for each variant, and then combines the results.

Is setting the TCP variant in scope of this helper, or outside scope?  I would think the latter.  That is, I think that existing facilities can be used to set the variant but the TcpTraceHelper code would be the same.  Please comment on this if someone disagrees.


> 
> Main points to consider in the implementation:
> 
> *) Right now all the applications create the socket inside their code, using
> an attribute to determine the transport type (i.e. UDP or TCP).
> 
> *) The sockets are created in the Start () method, so they don't exist
> before the start of the application

In general, one can't assume this.  For instance, a web server may create sockets dynamically during a run.

> 
> *) Saving all the data in RAM could be heavy; we need some class that
> offload to disk the data after some time

I think we can look at optimization as a second step.

> 
> My proposals:
> 
> *) Give a method to ALL applications for set an external-created socket. The
> helper will take take of creating sockets, installing it into applications
> (and attaching to trace sources). Example:
> 
> TcpTraceHelper tcpHelper;
> ...
> tcpHelper.CreateSocketFor (appContainer); /* for each app in apps; do
> Ptr<Socket> *s = CreateSocket; s->Connect... ; app->SetSocket (s); done */
> ...

I am not so much in favor of this design (see dynamic socket creation comment above).

I was thinking instead of adding a callback to TcpL4Protocol to notify of new socket creation, and let the helper register this callback, and let the helper hook sources upon callback invocation, provided that the socket matched the user-provided configuration for what socket 5 tuples were of interest.

e.g.

TcpTraceHelper tcpHelper;
NodeContainer nc = ...;
tcpHelper.Attach (nc);
tcpHelper.Trace (/* some descriptors including wildcards for the IP addresses and port numbers of interest; others that do not match will be ignored */);
tcpHelper.ConfigureOutput (/* some arguments to configure what style output is produced */);

with the goal that user should be able to specify the common case very simply in the API; in words, it may be:

"I want a gnuplot of cwnd evolution of all TCP flows matching the client IP/port of 192.168.0.100:* and server IP of 192.168.0.1:443, and the plot should be called 'tcp-cwnd-evolution.eps'"

> 
> *) Define a easy and simple language for allowing users to select 0 or more
> trace source to attach, with a way to identify flows.
> 
> Any help or hint is appreciated.

I also think that if we can use the data collection framework or Gnuplot class for the trace sink aspect of this, it will help.

(In reply to Tom Henderson from comment #2)
> Is setting the TCP variant in scope of this helper, or outside scope?  I
> would think the latter.  That is, I think that existing facilities can be
> used to set the variant but the TcpTraceHelper code would be the same. 
> Please comment on this if someone disagrees.

Sorry for not being exhaustive (I don't have my laptop). I believe that is in scope, since one of the classical needs is to compare different congestion control in the same simulation, ie is cubic friendly over newreno on such architecture? Is a single MyWonderfulTCP flow stealing all the bandwidth from others? Currently tcp-variants-comparison isn't able to do this. 

I like the callback-based idea; i will elaborate that. I'm also strongly in favor of using the statistic framework.

(In reply to natale.patriciello from comment #3)
> (In reply to Tom Henderson from comment #2)
> > Is setting the TCP variant in scope of this helper, or outside scope?  I
> > would think the latter.  That is, I think that existing facilities can be
> > used to set the variant but the TcpTraceHelper code would be the same. 
> > Please comment on this if someone disagrees.
> 
> Sorry for not being exhaustive (I don't have my laptop). I believe that is
> in scope, since one of the classical needs is to compare different
> congestion control in the same simulation, ie is cubic friendly over newreno
> on such architecture? Is a single MyWonderfulTCP flow stealing all the
> bandwidth from others? Currently tcp-variants-comparison isn't able to do
> this. 

I am not sure how to do what you ask within a traditional helper.  It is more like a 'runner' in that you want it to Simulator::Run(); Simulator::Destroy::(); multiple times, one for each TCP type.  That is why I'm wondering if this isn't better suited for some kind of external runner script.

Sorry for not answering, bugzilla apparently does not CC automatically :/

I like very much the callback on TcpL4Protocol. Providing a generic API might be too complex, we should let the user define a function returning a boolean that decides if the socket should be traced or not (and maybe also returns a mask of parameters to trace or/an helper).

I suggested adding a TcpTraceHelper because doing it manually is tedious, there are many parameters to hook to (Cwnd, Rcv Buffer etc...), peculiar cases to handle (you have to save the initial parameter values yourself). I was just looking at a way to drop all this data to then allow for postprocessing:
- create a folder for the connection and drop every parameter in a .csv there
- or alternatively create a csv/sql file with each line listing all parameters.

In general I believe it's best to compose components, i.e., not use the ns3 statistical framework. For instance I use the excellent "pandas" library to postprocess my data, some may prefer to use gnuplot (there are masochists everywhere :) ).

(In reply to Tom Henderson from comment #4)
> I am not sure how to do what you ask within a traditional helper.  It is
> more like a 'runner' in that you want it to Simulator::Run();
> Simulator::Destroy::(); multiple times, one for each TCP type.  That is why
> I'm wondering if this isn't better suited for some kind of external runner
> script.

My idea is to plan at the beginning all the socket properties such as congestion control (e.g. a list of string) and then, for each created socket, apply incrementally the property.

I don't know how to make TCP sockets interact between themselves with different run, launched with an external script. Without the interaction between flows, the entire concept of TCP fairness could not be investigated with ns-3.

(In reply to Matthieu Coudron from comment #5)
> In general I believe it's best to compose components, i.e., not use the ns3
> statistical framework. For instance I use the excellent "pandas" library to
> postprocess my data, some may prefer to use gnuplot (there are masochists
> everywhere :) ).

The purpose of the stats module is to automate the data saving (e.g. connect to the rtt trace source, and saving a pair <time measurement, rtt value>). How to print and then process this data is left to the researcher: it means that you can print the DataSet over a txt file, or to a sql database later (I'm not sure about the last part, since I always used some dirty hacks to print the dataset to a txt file)

I am refactoring a bit of mptcp code 'cos plotting the statistics is a bit of a nightmare.
I've added a simple callback
===
m_sockets.push_back (socket);
  
  if(!m_onNewSocket.IsNull())
  {
    NS_LOG_DEBUG ("Calling m_onNewSocket");
    m_onNewSocket (socket);
  }
===
from which I enable tracing for TCP socket. 
My question is the following: is it more desirable to output statistics to a single file (which I prefer) or create one file per attribute (consume more file descriptors, which can be annoying if there are many traced sockets)

Currently I have:
==
  AsciiTraceHelper asciiTraceHelper;
  Ptr<OutputStreamWrapper> streamTxNext = asciiTraceHelper.CreateFileStream (prefix+"_TxNext.csv", mode);
  Ptr<OutputStreamWrapper> streamTxHighest = asciiTraceHelper.CreateFileStream (prefix+"_TxHighest.csv", mode);
  Ptr<OutputStreamWrapper> streamRxAvailable = asciiTraceHelper.CreateFileStream (prefix+"_RxAvailable.csv", mode);
  Ptr<OutputStreamWrapper> streamRxTotal = asciiTraceHelper.CreateFileStream (prefix+"_RxTotal.csv", mode);
  Ptr<OutputStreamWrapper> streamRxNext = asciiTraceHelper.CreateFileStream (prefix+"_RxNext.csv", mode);
  Ptr<OutputStreamWrapper> streamTxUnack = asciiTraceHelper.CreateFileStream (prefix+"_TxUnack.csv", mode);
  Ptr<OutputStreamWrapper> streamStates = asciiTraceHelper.CreateFileStream (prefix+"_states.csv", mode);
  Ptr<OutputStreamWrapper> streamCwnd = asciiTraceHelper.CreateFileStream (prefix+"_cwnd.csv", mode);
  Ptr<OutputStreamWrapper> streamRwnd = asciiTraceHelper.CreateFileStream (prefix+"_rwnd.csv", mode);
  Ptr<OutputStreamWrapper> streamSSThreshold = asciiTraceHelper.CreateFileStream (prefix+"_ssThresh.csv", mode);
===

(In reply to Matthieu Coudron from comment #8)
> I am refactoring a bit of mptcp code 'cos plotting the statistics is a bit
> of a nightmare.
> I've added a simple callback
> ===
> m_sockets.push_back (socket);
>   
>   if(!m_onNewSocket.IsNull())
>   {
>     NS_LOG_DEBUG ("Calling m_onNewSocket");
>     m_onNewSocket (socket);
>   }
> ===
> from which I enable tracing for TCP socket. 
> My question is the following: is it more desirable to output statistics to a
> single file (which I prefer) or create one file per attribute (consume more
> file descriptors, which can be annoying if there are many traced sockets)

I tend to prefer one file; however, can you sketch out the layout you prefer for your single file?

moved to https://gitlab.com/nsnam/ns-3-dev/-/issues/192