Difference between revisions of "Statistical Framework for Network Simulation"

From Nsnam
Jump to: navigation, search
m
m
Line 4: Line 4:
 
A variety of these are [http://nsnam.isi.edu/nsnam/index.php/Contributed_Code#Support listed in the ns-2 wiki].  The following are notes on particular efforts.
 
A variety of these are [http://nsnam.isi.edu/nsnam/index.php/Contributed_Code#Support listed in the ns-2 wiki].  The following are notes on particular efforts.
  
* '''[http://info.iet.unipi.it/~cng/ns2measure/ ns2measure].'''  Main component is a global observer object integrated into ns-2.  Several generic types of measures are supported, e.g. time averaged and discrete rate.  Observed samples are recorded via an explicit call to the observer object, identified by a measure label and particular identifier such as a flow or host.  Post-simulation scripts provide for analyzing collected data, generating statistics.  Includes support for incorporating data from independent runs and for conducting runs until a statistical goal such as a confidence is met, but does not provide support for distributing execution of those runs.
+
* '''[http://info.iet.unipi.it/~cng/ns2measure/ ns2measure]''' provides a data collection framework for ns-2 and support for calculating statistics over that data, including multiple runs. The main component is a global observer object incorporated into ns-2.  Several generic types of measures are supported, e.g. time averaged and discrete rate.  Observed samples are recorded via an explicit call to the observer object, identified by a measure label and particular identifier such as a flow or host.  Post-simulation scripts provide for analyzing collected data and generating statistics.  A control script is provided such that runs may be repeated until a statistical goal such as a confidence level is met.  Data from independent runs may be incorporated in generation of the statistics.
* '''[http://www.wand.net.nz/~stj2/nsc/software.html simd]''' provides support for executing distributed, independent simulation runs and collecting data from them.  A set of python scripts is used to push simulations out to client nodes, with a standardized set of scripts used to parameterize runs.  Scripts are expected to produce output as comma seperated values, which are collected and concatenated by the master control script.
+
* '''[http://www.wand.net.nz/~stj2/nsc/software.html simd]''' executes distributed, independent simulation runs and collects data from them.  A set of python scripts is used to push simulations out to client nodes, with a standardized set of scripts used to parameterize runs.  Scripts are expected to produce output as comma seperated values, which are collected and concatenated by the master control script.
* '''[http://www-tkn.ee.tu-berlin.de/research/ns-2_akaroa-2/ns.html ns-2/akaroa-2]''' provides support for executing distributed, independent replications, with significant statistical support for working with collected data and managing the runs.  A master program runs on one computer and a set of clients on other machines that execute received simulations.  Within each ns-2 instance a global observer is created.  Samples are reported to that observer, which forwards them to the master computer.  Measures are identified in simulation scripts by numeric identifiers and consist of particular observations, e.g. delay or packet size.  The master program receives these observations and calculates statistics such as the mean and confidence interval over them.  That data is used both for final output, and to conduct more simulations at the client machines if confidence is low.  Another change to ns-2 is incorporation of a different random number generator with better guarantees for independent streams.   
+
* '''[http://www-tkn.ee.tu-berlin.de/research/ns-2_akaroa-2/ns.html ns-2/akaroa-2]''' provides support for executing distributed, independent replications, with significant statistical support for working with collected data and managing the runs.  A master program runs on one computer and a set of clients on other machines that execute received simulations.  Within each ns-2 instance a global observer is created.  Samples are reported to that observer, which forwards them to the master computer.  Measures are identified in simulation scripts by numeric identifiers and consist of particular observations, e.g. delay or packet size.  The master program receives these observations and calculates statistics such as the mean and confidence interval over them.  That data is used both for final output, and to conduct more simulations at the client machines if confidence is low.  Another addition to ns-2 is incorporation of a different random number generator with better guarantees for independent streams.   
 
* '''[http://www.tracegraph.com/ tracegraph].'''
 
* '''[http://www.tracegraph.com/ tracegraph].'''
 
* '''[http://networks.ecse.rpi.edu/~harrisod/graph.html rpi ns2graph].'''
 
* '''[http://networks.ecse.rpi.edu/~harrisod/graph.html rpi ns2graph].'''

Revision as of 19:20, 16 May 2008

Previous Work

Several components and packages have been made for ns-2 to collect and manage data and statistics. A variety of these are listed in the ns-2 wiki. The following are notes on particular efforts.

  • ns2measure provides a data collection framework for ns-2 and support for calculating statistics over that data, including multiple runs. The main component is a global observer object incorporated into ns-2. Several generic types of measures are supported, e.g. time averaged and discrete rate. Observed samples are recorded via an explicit call to the observer object, identified by a measure label and particular identifier such as a flow or host. Post-simulation scripts provide for analyzing collected data and generating statistics. A control script is provided such that runs may be repeated until a statistical goal such as a confidence level is met. Data from independent runs may be incorporated in generation of the statistics.
  • simd executes distributed, independent simulation runs and collects data from them. A set of python scripts is used to push simulations out to client nodes, with a standardized set of scripts used to parameterize runs. Scripts are expected to produce output as comma seperated values, which are collected and concatenated by the master control script.
  • ns-2/akaroa-2 provides support for executing distributed, independent replications, with significant statistical support for working with collected data and managing the runs. A master program runs on one computer and a set of clients on other machines that execute received simulations. Within each ns-2 instance a global observer is created. Samples are reported to that observer, which forwards them to the master computer. Measures are identified in simulation scripts by numeric identifiers and consist of particular observations, e.g. delay or packet size. The master program receives these observations and calculates statistics such as the mean and confidence interval over them. That data is used both for final output, and to conduct more simulations at the client machines if confidence is low. Another addition to ns-2 is incorporation of a different random number generator with better guarantees for independent streams.
  • tracegraph.
  • rpi ns2graph.
  • ns2 jtrana takes an ns-2 trace file, parses it into MySQL, and provides an interface to interrogate the database and produce graphs and other output in several formats. The database scheme is largely a straightforward encoding of the raw trace data.
  • Samer Bali's scripts implement post-simulation trace analysis, with support for averaging from multiple runs. Measures calculated include packet delivery ratio, normalized throughput, routing overhead ratio, end to end delay, hop count, and connection time.

Raw Notes

fault tolerance