NSF Frameworks - Revision history

Tomh at 00:17, 28 February 2015

2015-02-28T00:17:59Z

← Older revision		Revision as of 00:17, 28 February 2015
Line 49:		Line 49:
	=== Maintenance ===		=== Maintenance ===

	This topic was led by [http://www.ee.washington.edu/faculty/affiliate_henderson.html Tom Henderson].		This topic was led by [http://www.ee.washington.edu/faculty/affiliate_henderson.html Tom Henderson]. Tom Henderson, John Abraham, Brian Swenson, and Daniel Lertpratchya collaborated to make ns-3 releases and maintain the ns-3 project infrastructure.

	== Publications ==		== Publications ==

	The following publications were partially funded by this project:		The following publications were partially funded by this project:

			* Data Visualization for Network Simulations, Christopher S. Main, L. Felipe Perrone, and Greg L. Schrock, Winter Simulation Conference 2014. Savannah, GA.

Tomh: update nsf frameworks page

2015-02-27T23:59:47Z

update nsf frameworks page

← Older revision		Revision as of 23:59, 27 February 2015
Line 1:		Line 1:
	{{TOC}}		{{TOC}}

	The U.S. National Science Foundation (NSF) ~~is funding~~ the University of Washington (Tom Henderson), Georgia Institute of Technology (George Riley), and Bucknell University (Felipe Perrone) to develop extensions to ns-3 to support the creation of more rigorous simulation studies and to support the ongoing maintenance of the ns-3 project. The project ~~is scheduled~~ to ~~run between 2010 and 2014~~. Below are ~~listed some near- and long-term plans in each area~~.		The U.S. National Science Foundation (NSF) has funded the University of Washington (Tom Henderson), Georgia Institute of Technology (George Riley), and [http://www.nsf.gov/awardsearch/showAward?AWD_ID=0958142 Bucknell University (Felipe Perrone)] to develop extensions to ns-3 to support the creation of more rigorous simulation studies and to support the ongoing maintenance of the ns-3 project. The project ran from 2010 to 2015. Below are the project outcomes.

	= Overview =		== Overview ==
	~~== Automation framework~~ ==

	This ~~work is~~ led by [http://www.eg.bucknell.edu/~perrone/Welcome.html Felipe Perrone].		The project performed work on three topics: 1) Automation framework, 2) Scenario Development, and 3) Maintenance.

			=== Automation framework ===

			This topic was led by [http://www.eg.bucknell.edu/~perrone/Welcome.html Felipe Perrone].

	'''Project goal:''' The automation framework will consist of user interfaces, description languages, and tools that will help		'''Project goal:''' The automation framework will consist of user interfaces, description languages, and tools that will help
Line 12:		Line 15:
	offered will enable the user to define, deploy, and control ns-3 simulation experiments that are methodologically valid and easy to reproduce by third parties. The framework will include tools for: model composition; structural validation of the model; configuration of model components; description, deployment, and control of experiments; output data processing and storage; and reporting of experimental setup. Although the framework will offer graphical user interfaces, more experienced users will be able to access automation functionality via the command-line.		offered will enable the user to define, deploy, and control ns-3 simulation experiments that are methodologically valid and easy to reproduce by third parties. The framework will include tools for: model composition; structural validation of the model; configuration of model components; description, deployment, and control of experiments; output data processing and storage; and reporting of experimental setup. Although the framework will offer graphical user interfaces, more experienced users will be able to access automation functionality via the command-line.

	Additional information is located [http://redmine.eg.bucknell.edu/safe ~~here~~]. ~~Much of the code under development~~ is available in the [http://code.nsnam.org/safe/ns-3-safe SAFE repository].		Additional information is located [http://redmine.eg.bucknell.edu/safe at a Bucknell Redmine site] and [http://redmine.eg.bucknell.edu/perrone/projects/framework/repository Bucknell code repository]. Code intended for transition to ns-3 is available in the [http://code.nsnam.org/safe/ns-3-safe SAFE repository] or other repositories under the username 'safe' at the ns-3 code server.

			The topic had four subtasks; below are links to the output of each subtask.

	~~A number of efforts are being worked in parallel:~~		==== User interfaces ====
	* A Data Collection Framework (DCF) is being developed to make it easier to get data out of the simulator and into plots and databases.
	* Output data analyzers are being constructed to detect steady-state and termination criteria in the output data stream.
	* An Experiment Execution Manager is being developed to manage multiple independent replications of simulation experiments across single or multiple machines.
	* Web-based simulation configuration and data visualization APIs are under construction.

	== ~~Scenario generation~~ ==		==== Experiment management ====

	~~This work is led by [http://www.ece.gatech.edu/faculty-staff/fac_profiles/bio.php?id~~=~~86 George Riley]~~		==== Simulation control ====

	~~'''Project goal:''' Random topology generation based on empirical observations~~ of ~~the Internet (e.g. BRITE), recreated topologies based on empirical data (e.g. Routeviews, Rocketfuel), and synthetic topology generation based on graph structures.~~		==== Output processing ====

			==== Verifying completeness and consistency of models ====

			==== Generation of simulator specific scripts ====

	'''Recent work:''' [http://www.cs.bu.edu/brite/ BRITE] integration with ns-3 was completed for the ns-3.16 release. Earlier work on the use of stock topology objects (e.g. point-to-point and dumbbell helpers) was performed under this task.		=== Scenario generation ===

	~~'''Near term plans~~:~~''' Software to construct AS topology graphs based on annotated network graphs~~		This topic was led by [http://www.ece.gatech.edu/faculty-staff/fac_profiles/bio.php?id=86 George Riley]

	~~== Educational framework ==~~		'''Project goal:''' Random topology generation based on empirical observations of the Internet (e.g. BRITE), recreated topologies based on empirical data (e.g. Routeviews, Rocketfuel), and synthetic topology generation based on graph structures.

	~~'''Project goal:''' Develop an on-line library of educational programs and laboratory experiments, and the supporting web-based framework~~ to ~~solicit additional contributions from~~ the ~~educational community~~.		The topic had four subtasks; below are links to the outcomes of each subtask.

	~~'''Recent work:''' The [[NetAnim \| NetAnim animator]] continues to be used to support courseware in a networking course at Georgia Tech.~~		==== Artificially Created Topologies ====

	~~'''Near term plans:''' Enhance the [[Education]] page and available scripts.~~		==== Random Topology Generation ====

	= ~~Prototype of automation framework~~ =		==== Empirical Topology Generation ====

	~~The project is working on a prototype that will integrate initial pieces of the automation framework, including the following:~~		==== Mobility Scenarios ====
	~~# a data collection framework to extract the data of interest, perhaps requiring some additions to how this data is accessed from the routing protocol implementations~~
	~~# support to easily plot the data points using confidence intervals using a plotting program such as gnuplot or matplotlib~~
	~~# ability to archive the complete state of the experiment so that it can be reproduced many years later~~
	~~# a steady-state detector to look for reaching the time to start data collection (and data deletion prior to that time)~~
	~~# a termination detector to terminate the program once the desired number of samples~~
	~~# an experiment execution manager (outside of ns-3) to manage the serial or parallel execution of simulation runs to obtain data points for each configuration~~
	~~# a basic wireless/mobility artificial scenario generator allowing the user to rerun the experiment with different numbers of nodes, node densities, and rate of link connectivity changes~~

	~~A stretch goal is to allow users to plot characteristics of the scenario according to a "god-like" view of the topology, using certain assumptions, such as:~~		=== Maintenance ===
	~~# true shortest paths available to each application packet originated over all scenarios~~
	~~# counts on the number of link connectivity changes~~

	~~Some elements of the above support are expected to be merged with ns-3, and some will live in an outside framework that can be downloaded as an extension via download~~.~~py or equivalent~~.		This topic was led by [http://www.ee.washington.edu/faculty/affiliate_henderson.html Tom Henderson].

	A concrete example to work is based on the program examples/wireless/manet-routing-compare.cc, which is based on the scenario described in the 1998 Mobicom paper by Broch et al. "A Performance Comparison of Multi-Hop Wireless Ad Hoc Network Routing Protocols". In particular, we would like the prototype to be able to generate plots such as figures 2 and 3 of that paper, and (for stretch goals) figure 1 and table V.		== Publications ==

	~~Code will be maintained at~~:		The following publications were partially funded by this project:
	* [http://code.nsnam.org/safe/ns-3-safe ns-3 safe repository]
	* [http://redmine.eg.bucknell.edu/perrone/projects/framework/repository Bucknell SAFE repository]

Tomh: update url

2013-02-06T13:41:38Z

update url

← Older revision		Revision as of 13:41, 6 February 2013
Line 12:		Line 12:
	offered will enable the user to define, deploy, and control ns-3 simulation experiments that are methodologically valid and easy to reproduce by third parties. The framework will include tools for: model composition; structural validation of the model; configuration of model components; description, deployment, and control of experiments; output data processing and storage; and reporting of experimental setup. Although the framework will offer graphical user interfaces, more experienced users will be able to access automation functionality via the command-line.		offered will enable the user to define, deploy, and control ns-3 simulation experiments that are methodologically valid and easy to reproduce by third parties. The framework will include tools for: model composition; structural validation of the model; configuration of model components; description, deployment, and control of experiments; output data processing and storage; and reporting of experimental setup. Although the framework will offer graphical user interfaces, more experienced users will be able to access automation functionality via the command-line.

	Additional information is located [http://redmine.eg.bucknell.edu/~~perrone/projects/framework/wiki~~ here]. Much of the code under development is available in the [http://code.nsnam.org/safe/ns-3-safe SAFE repository].		Additional information is located [http://redmine.eg.bucknell.edu/safe here]. Much of the code under development is available in the [http://code.nsnam.org/safe/ns-3-safe SAFE repository].

	A number of efforts are being worked in parallel:		A number of efforts are being worked in parallel:

Tomh: /* Scenario generation */

2013-02-05T14:53:25Z

Scenario generation

← Older revision		Revision as of 14:53, 5 February 2013
Line 26:		Line 26:
	'''Project goal:''' Random topology generation based on empirical observations of the Internet (e.g. BRITE), recreated topologies based on empirical data (e.g. Routeviews, Rocketfuel), and synthetic topology generation based on graph structures.		'''Project goal:''' Random topology generation based on empirical observations of the Internet (e.g. BRITE), recreated topologies based on empirical data (e.g. Routeviews, Rocketfuel), and synthetic topology generation based on graph structures.

	'''Recent work:''' [http://www.cs.bu.edu/brite/ BRITE] integration with ns-3 was completed for the ns-3.16 release. ~~Work~~ on the use of		'''Recent work:''' [http://www.cs.bu.edu/brite/ BRITE] integration with ns-3 was completed for the ns-3.16 release. Earlier work on the use of stock topology objects (e.g. point-to-point and dumbbell helpers) was performed under this task.

	'''Near term plans:''' Software to construct AS topology graphs based on annotated network graphs		'''Near term plans:''' Software to construct AS topology graphs based on annotated network graphs

Tomh: /* Scenario generation */

2013-02-05T14:52:37Z

Scenario generation

← Older revision		Revision as of 14:52, 5 February 2013
Line 28:		Line 28:
	'''Recent work:''' [http://www.cs.bu.edu/brite/ BRITE] integration with ns-3 was completed for the ns-3.16 release. Work on the use of		'''Recent work:''' [http://www.cs.bu.edu/brite/ BRITE] integration with ns-3 was completed for the ns-3.16 release. Work on the use of

	'''Near term plans:''' ~~[http://www.cs.bu.edu/brite/ BRITE] integration with ns-3 was completed for the ns-3.16 release.~~		'''Near term plans:''' Software to construct AS topology graphs based on annotated network graphs

	== Educational framework ==		== Educational framework ==

Tomh: /* Prototype */

2013-02-05T14:51:36Z

Prototype

← Older revision		Revision as of 14:51, 5 February 2013
Line 38:		Line 38:
	'''Near term plans:''' Enhance the [[Education]] page and available scripts.		'''Near term plans:''' Enhance the [[Education]] page and available scripts.

	= Prototype =		= Prototype of automation framework =

	The project is working on a prototype that will integrate initial pieces of the automation framework, including the following:		The project is working on a prototype that will integrate initial pieces of the automation framework, including the following:

Tomh: /* Overview */

2013-02-05T14:51:04Z

Overview

← Older revision		Revision as of 14:51, 5 February 2013
Line 12:		Line 12:
	offered will enable the user to define, deploy, and control ns-3 simulation experiments that are methodologically valid and easy to reproduce by third parties. The framework will include tools for: model composition; structural validation of the model; configuration of model components; description, deployment, and control of experiments; output data processing and storage; and reporting of experimental setup. Although the framework will offer graphical user interfaces, more experienced users will be able to access automation functionality via the command-line.		offered will enable the user to define, deploy, and control ns-3 simulation experiments that are methodologically valid and easy to reproduce by third parties. The framework will include tools for: model composition; structural validation of the model; configuration of model components; description, deployment, and control of experiments; output data processing and storage; and reporting of experimental setup. Although the framework will offer graphical user interfaces, more experienced users will be able to access automation functionality via the command-line.

	~~Overall development roadmap~~ is ~~listed~~ [http://redmine.eg.bucknell.edu/perrone/projects/framework/wiki here].		Additional information is located [http://redmine.eg.bucknell.edu/perrone/projects/framework/wiki here]. Much of the code under development is available in the [http://code.nsnam.org/safe/ns-3-safe SAFE repository].

			A number of efforts are being worked in parallel:
			* A Data Collection Framework (DCF) is being developed to make it easier to get data out of the simulator and into plots and databases.
			* Output data analyzers are being constructed to detect steady-state and termination criteria in the output data stream.
			* An Experiment Execution Manager is being developed to manage multiple independent replications of simulation experiments across single or multiple machines.
			* Web-based simulation configuration and data visualization APIs are under construction.

	== Scenario generation ==		== Scenario generation ==
Line 20:		Line 26:
	'''Project goal:''' Random topology generation based on empirical observations of the Internet (e.g. BRITE), recreated topologies based on empirical data (e.g. Routeviews, Rocketfuel), and synthetic topology generation based on graph structures.		'''Project goal:''' Random topology generation based on empirical observations of the Internet (e.g. BRITE), recreated topologies based on empirical data (e.g. Routeviews, Rocketfuel), and synthetic topology generation based on graph structures.

	'''Near term plans:''' [http://www.cs.bu.edu/brite/ BRITE] integration with ns-3 ~~scheduled~~ for ~~fall of 2010~~.		'''Recent work:''' [http://www.cs.bu.edu/brite/ BRITE] integration with ns-3 was completed for the ns-3.16 release. Work on the use of

			'''Near term plans:''' [http://www.cs.bu.edu/brite/ BRITE] integration with ns-3 was completed for the ns-3.16 release.

	== Educational framework ==		== Educational framework ==
Line 26:		Line 34:
	'''Project goal:''' Develop an on-line library of educational programs and laboratory experiments, and the supporting web-based framework to solicit additional contributions from the educational community.		'''Project goal:''' Develop an on-line library of educational programs and laboratory experiments, and the supporting web-based framework to solicit additional contributions from the educational community.

	'''~~Near term plans~~:''' ~~George Riley and Josh Pelkey~~ used ~~ns-3 as~~ courseware in a ~~fall 2010 semester~~ course at Georgia Tech. ~~Materials from this course will form our initial contributions to the educational program repository.~~		'''Recent work:''' The [[NetAnim \| NetAnim animator]] continues to be used to support courseware in a networking course at Georgia Tech.

	~~Lalith Suresh~~'s [[~~NS3_Lab_Assignments~~]] page.		'''Near term plans:''' Enhance the [[Education]] page and available scripts.

	= Prototype =		= Prototype =

Tomh: add info about prototype goals

2011-06-29T04:17:36Z

add info about prototype goals

@@ Line 3: / Line 3: @@
 The U.S. National Science Foundation (NSF) is funding the University of Washington (Tom Henderson), Georgia Institute of Technology (George Riley), and Bucknell University (Felipe Perrone) to develop extensions to ns-3 to support the creation of more rigorous simulation studies and to support the ongoing maintenance of the ns-3 project.  The project is scheduled to run between 2010 and 2014.  Below are listed some near- and long-term plans in each area.
-= Automation framework =
+= Overview =
 This work is led by [http://www.eg.bucknell.edu/~perrone/Welcome.html Felipe Perrone].
@@ Line 13: / Line 14: @@
 Overall development roadmap is listed [http://redmine.eg.bucknell.edu/perrone/projects/framework/wiki here].
-= Scenario generation =
+== Scenario generation ==
 This work is led by [http://www.ece.gatech.edu/faculty-staff/fac_profiles/bio.php?id=86 George Riley]
@@ Line 21: / Line 22: @@
 '''Near term plans:''' [http://www.cs.bu.edu/brite/ BRITE] integration with ns-3 scheduled for fall of 2010.
-= Educational framework =
+== Educational framework ==
 '''Project goal:'''  Develop an on-line library of educational programs and laboratory experiments, and the supporting web-based framework to solicit additional contributions from the educational community.
@@ Line 28: / Line 29: @@
 Lalith Suresh's [[NS3_Lab_Assignments]] page.

Tomh: update frameworks wiki

2011-06-28T18:52:59Z

update frameworks wiki

← Older revision		Revision as of 18:52, 28 June 2011
Line 10:		Line 10:
	users of varying levels of expertise to produce more credible simulation experiments with ns-3. The functionality		users of varying levels of expertise to produce more credible simulation experiments with ns-3. The functionality
	offered will enable the user to define, deploy, and control ns-3 simulation experiments that are methodologically valid and easy to reproduce by third parties. The framework will include tools for: model composition; structural validation of the model; configuration of model components; description, deployment, and control of experiments; output data processing and storage; and reporting of experimental setup. Although the framework will offer graphical user interfaces, more experienced users will be able to access automation functionality via the command-line.		offered will enable the user to define, deploy, and control ns-3 simulation experiments that are methodologically valid and easy to reproduce by third parties. The framework will include tools for: model composition; structural validation of the model; configuration of model components; description, deployment, and control of experiments; output data processing and storage; and reporting of experimental setup. Although the framework will offer graphical user interfaces, more experienced users will be able to access automation functionality via the command-line.

	~~'''Near term plans:'''~~

	Work is underway on an [http://redmine.eg.bucknell.edu/perrone/projects/framework/wiki/Execution_manager execution manager] and a [http://redmine.eg.bucknell.edu/perrone/projects/framework/wiki/Samples_of_the_metrics_of_interest data collection framework].

	Overall development roadmap is listed [http://redmine.eg.bucknell.edu/perrone/projects/framework/wiki here].		Overall development roadmap is listed [http://redmine.eg.bucknell.edu/perrone/projects/framework/wiki here].
Line 29:		Line 25:
	'''Project goal:''' Develop an on-line library of educational programs and laboratory experiments, and the supporting web-based framework to solicit additional contributions from the educational community.		'''Project goal:''' Develop an on-line library of educational programs and laboratory experiments, and the supporting web-based framework to solicit additional contributions from the educational community.

	'''Near term plans:''' George Riley and Josh Pelkey ~~will be using~~ ns-3 as courseware in a fall semester course at Georgia Tech. Materials from this course will form our initial contributions to the educational program repository.		'''Near term plans:''' George Riley and Josh Pelkey used ns-3 as courseware in a fall 2010 semester course at Georgia Tech. Materials from this course will form our initial contributions to the educational program repository.

			Lalith Suresh's [[NS3_Lab_Assignments]] page.

Tomh: frameworks for ns-3 update

2010-08-27T22:11:17Z

frameworks for ns-3 update

← Older revision		Revision as of 22:11, 27 August 2010
Line 11:		Line 11:
	offered will enable the user to define, deploy, and control ns-3 simulation experiments that are methodologically valid and easy to reproduce by third parties. The framework will include tools for: model composition; structural validation of the model; configuration of model components; description, deployment, and control of experiments; output data processing and storage; and reporting of experimental setup. Although the framework will offer graphical user interfaces, more experienced users will be able to access automation functionality via the command-line.		offered will enable the user to define, deploy, and control ns-3 simulation experiments that are methodologically valid and easy to reproduce by third parties. The framework will include tools for: model composition; structural validation of the model; configuration of model components; description, deployment, and control of experiments; output data processing and storage; and reporting of experimental setup. Although the framework will offer graphical user interfaces, more experienced users will be able to access automation functionality via the command-line.

	'''Near term plans:''' http://redmine.eg.bucknell.edu/perrone/projects/framework		'''Near term plans:'''

			Work is underway on an [http://redmine.eg.bucknell.edu/perrone/projects/framework/wiki/Execution_manager execution manager] and a [http://redmine.eg.bucknell.edu/perrone/projects/framework/wiki/Samples_of_the_metrics_of_interest data collection framework].

			Overall development roadmap is listed [http://redmine.eg.bucknell.edu/perrone/projects/framework/wiki here].

	= Scenario generation =		= Scenario generation =
Line 25:		Line 29:
	'''Project goal:''' Develop an on-line library of educational programs and laboratory experiments, and the supporting web-based framework to solicit additional contributions from the educational community.		'''Project goal:''' Develop an on-line library of educational programs and laboratory experiments, and the supporting web-based framework to solicit additional contributions from the educational community.

	'''Near term plans:''' ~~None yet~~.		'''Near term plans:''' George Riley and Josh Pelkey will be using ns-3 as courseware in a fall semester course at Georgia Tech. Materials from this course will form our initial contributions to the educational program repository.