Difference between revisions of "App Store Technical Requirements"

From Nsnam
Jump to: navigation, search
(update page)
(update app store requirements)
Line 1: Line 1:
 
{{TOC}}
 
{{TOC}}
  
= Goals =
+
= Background =
  
The long-term goal is to move ns-3 to separate modules, for build and  
+
The goal is to support modularization of the ns-3 code base and federation of development activitiesTo date, the main path to distributing extensions to ns-3 has been to request a code review for merge to the mainline tree.  The future goal is to shrink the ns-3 core, and to better facilitate modules that are independently developed and maintained.  As a concrete example, the [https://www.r-project.org/ R statistical framework] does not integrate every extension to its main release; instead, it allows users to [https://cran.cnr.berkeley.edu/web/packages/available_packages_by_date.html fetch the packages of interest] and add to their installationCommunities-of-interest have even arisen to allow researchers to organize and curate sets of R extensions (e.g. [https://www.bioconductor.org/ bioinformatic packages]).  We envision the same for ns-3.
maintenance reasonsFor build reasons, since ns-3 is becoming large,  
+
we would like to allow users to enable/disable subsets of the available
+
model libraryFor maintenance reasons, it is important that we move to  
+
a development model where modules can evolve on different timescales and  
+
be maintained by different organizations.
+
  
An analogy is the GNOME desktop, which is composed of a number of
+
We are taking steps, to be first enabled for ns-3.27, to allow third party contributed code modules to be added to ns-3, and to be downloaded, built, and installed by ns-3's bake package management toolThis will allow modules to be hosted elsewhere and integrated with the ns-3-allinone framework. Consider this to be the back-end support for modularization.
individual libraries that evolve on their own timescalesA build
+
framework called [http://live.gnome.org/Jhbuild jhbuild] exists for
+
building and managing the dependencies between these disparate projects.
+
  
The modular build and integration with bake is described in a separate [[BakeIntegration | wiki page]] and is intended to be enabled for ns-3.27.
+
For example, the sift routing code contribution has been added as a module.  Please review how users are expected to add sift to their ns-3.27 release [[BakeIntegration | on this wiki page]].
  
Once we have a modular build, and an ability to separately download and
+
What we seek in an app store is the web-based front end that allows users to discover packagesIt turns out that another open source project (Cytoscape) has already built such an app store that is basically what we seek; see the [http://apps.cytoscape.org/ Cytoscape app store]Moreover, the code to deploy this app store is open source, and is designed to work with Wordpress, meaning that we could, in principle, deploy an ns-3-centric instance of this.
install third-party modules, we will need to distinguish the maintenance
+
status or certification of modulesThe ns-3 project will maintain a
+
set of core ns-3 modules including those essential for all ns-3
+
simulations, and will maintain a master build file containing metadata
+
to contributed modules; this will allow users to fetch and build what
+
they needEventually, modules will have a maintenance state associated
+
with them describing aspects such as who is the maintainer, whether it
+
is actively maintained, whether it contains documentation or tests,  
+
whether it passed an ns-3 project code review, whether it is currently
+
passing the tests, etc.  The current status of all known ns-3 modules
+
will be maintained in a database and be browsable on the project web site.
+
  
[[image:Maintenance-status-example.PNG]]
+
= Links =
  
'''Figure caption''': ''Mock-up of future model status page (models and colors selected are just for example purposes)''
+
* [http://wiki.cytoscape.org/AppStore Cytoscape wiki]
 +
* [https://github.com/cytoscape/appstore Cytoscape app store code repository]
 +
* [https://github.com/cytoscape/appstore/wiki/Running-App-Store-on-Docker-Containers Cytoscape docker containers]
 +
 
 +
= Requirements =
  
The basic idea of the ns-3 app store would be to store on a server a set of user-submitted metadata which describes various source code packages. Typical metadata would include:
+
The main functional goal is to deploy something very close to the Cytoscape app store on the ns-3 web server. However, because ns-3 does source and not binary distributions, this tool will facilitate source code downloads of modules (apps).
* unique name
+
* version number
+
* last known good ns-3 version (tested against)
+
* description
+
* download url
+
* untar/unzip command
+
* configure command
+
* build command
+
* system prerequisites (if user needs to apt-get install other libraries)
+
* list of ns-3 package dependencies (other ns-3 packages which this package depends upon)
+
  
This page used to have a list of requirements for a home-grown app store, but lately, we are thinking of adapting the implementation at [http://apps.cytoscape.org/ Cytoscape] and tailoring it to ns-3's needs.  This is a project idea for the [https://www.nsnam.org/wiki/GSOC2017Projects#App_Store 2017 Google Summer of Code].
+
#. The front page should be clean, consisting of categories, newest releases, and popular downloads.
 +
#. Each "app" will be an ns-3 module, with a description page similar to Cytoscape's page:  Description, screenshots,  download tab, resources tab, etc.
 +
#. Clear documentation must be posted on how to use the app store, and how to submit a new app to the app store
 +
#. The "Download" tab will fetch a source archive, and the user will download to a location of their choice.  The user is responsible for moving or copying the package to their ns-3 build location, and then the ns-3 build tools (invoked at command line by users in the normal way) will do the rest.
 +
 
 +
= Enhancements =
 +
 
 +
Once the basic capability is deployed, enhancements are possible.  Some of these extensions may involve modifying the app store installation, but some will involve modifying ns-3 build scripts, documentation, and other scripts (e.g. Jenkins scripts).
 +
 
 +
== Integration with bake ==
 +
 
 +
Bake uses XML to describe modules.  It would be nice to have a standardized XML schema that describes an ns-3 module, and make it usable from both the app store and from bake.  For instance, there could be an "authors" field in the XML description that can be displayed by bake at the command line and can also be parsed and shown as the Authors in the app store description.  A Wordpress plugin might be able to accomplish this.
 +
 
 +
== User feedback and comments ==
 +
 
 +
We probably want to explore and tailor the capability for users to provide feedback and hints about apps.  We do not want the capability to be abused by spammers (e.g. a captcha?).
 +
 
 +
== Documentation and module creation scripts ==
 +
 
 +
* Doxygen and Sphinx scripts should be modified to allow building on a per-module basis; ns-3 web site should still integrate all of these module documentations coherently
 +
* The create-module.py program should be updated
 +
* The ns-3 tutorial must be updated
 +
 
 +
== Regression testing ==
 +
 
 +
We need to write some new [https://ns-buildmaster.ee.washington.edu:8010/ Jenkins scripts] to test the known modules for regressions.
 +
 
 +
== Versioning ==
 +
 
 +
Managing the version numbers of different apps will be necessary.  We want users to be able to declare that apps are compatible with specific ns-3 versions.
 +
 
 +
== Patch management ==
 +
 
 +
Apps may fall out of date (become unmaintained) but it may be the case that some simple patches could be applied on top of them to make them workable again with the latest ns-3.  The framework should support the ability to declare that a module installation consists of downloading an archive and then patching it with a file fetched from elsewhere.  A module may also require patching of ns-3 core itself; the framework should support this.
 +
 
 +
== Dependency management ==
 +
 
 +
If some modules depend on other modules or system dependencies, that needs to be expressed and handled by the tools.  This is mainly a bake build issue, but it probably requires also some documentation in the app store that the module depends, for example, on a system library such as Boost.
 +
 
 +
== Module grouping ==
 +
 
 +
We will probably want support for groups of modules.  Users should be able to specify, for example, "Ad hoc routing" and fetch modules for several protocols, which may be hosted at different sites.

Revision as of 17:00, 29 March 2017

Main Page - Current Development - Developer FAQ - Tools - Related Projects - Project Ideas - Summer Projects

Installation - Troubleshooting - User FAQ - HOWTOs - Samples - Models - Education - Contributed Code - Papers

Background

The goal is to support modularization of the ns-3 code base and federation of development activities. To date, the main path to distributing extensions to ns-3 has been to request a code review for merge to the mainline tree. The future goal is to shrink the ns-3 core, and to better facilitate modules that are independently developed and maintained. As a concrete example, the R statistical framework does not integrate every extension to its main release; instead, it allows users to fetch the packages of interest and add to their installation. Communities-of-interest have even arisen to allow researchers to organize and curate sets of R extensions (e.g. bioinformatic packages). We envision the same for ns-3.

We are taking steps, to be first enabled for ns-3.27, to allow third party contributed code modules to be added to ns-3, and to be downloaded, built, and installed by ns-3's bake package management tool. This will allow modules to be hosted elsewhere and integrated with the ns-3-allinone framework. Consider this to be the back-end support for modularization.

For example, the sift routing code contribution has been added as a module. Please review how users are expected to add sift to their ns-3.27 release on this wiki page.

What we seek in an app store is the web-based front end that allows users to discover packages. It turns out that another open source project (Cytoscape) has already built such an app store that is basically what we seek; see the Cytoscape app store. Moreover, the code to deploy this app store is open source, and is designed to work with Wordpress, meaning that we could, in principle, deploy an ns-3-centric instance of this.

Links

Requirements

The main functional goal is to deploy something very close to the Cytoscape app store on the ns-3 web server. However, because ns-3 does source and not binary distributions, this tool will facilitate source code downloads of modules (apps).

  1. . The front page should be clean, consisting of categories, newest releases, and popular downloads.
  2. . Each "app" will be an ns-3 module, with a description page similar to Cytoscape's page: Description, screenshots, download tab, resources tab, etc.
  3. . Clear documentation must be posted on how to use the app store, and how to submit a new app to the app store
  4. . The "Download" tab will fetch a source archive, and the user will download to a location of their choice. The user is responsible for moving or copying the package to their ns-3 build location, and then the ns-3 build tools (invoked at command line by users in the normal way) will do the rest.

Enhancements

Once the basic capability is deployed, enhancements are possible. Some of these extensions may involve modifying the app store installation, but some will involve modifying ns-3 build scripts, documentation, and other scripts (e.g. Jenkins scripts).

Integration with bake

Bake uses XML to describe modules. It would be nice to have a standardized XML schema that describes an ns-3 module, and make it usable from both the app store and from bake. For instance, there could be an "authors" field in the XML description that can be displayed by bake at the command line and can also be parsed and shown as the Authors in the app store description. A Wordpress plugin might be able to accomplish this.

User feedback and comments

We probably want to explore and tailor the capability for users to provide feedback and hints about apps. We do not want the capability to be abused by spammers (e.g. a captcha?).

Documentation and module creation scripts

  • Doxygen and Sphinx scripts should be modified to allow building on a per-module basis; ns-3 web site should still integrate all of these module documentations coherently
  • The create-module.py program should be updated
  • The ns-3 tutorial must be updated

Regression testing

We need to write some new Jenkins scripts to test the known modules for regressions.

Versioning

Managing the version numbers of different apps will be necessary. We want users to be able to declare that apps are compatible with specific ns-3 versions.

Patch management

Apps may fall out of date (become unmaintained) but it may be the case that some simple patches could be applied on top of them to make them workable again with the latest ns-3. The framework should support the ability to declare that a module installation consists of downloading an archive and then patching it with a file fetched from elsewhere. A module may also require patching of ns-3 core itself; the framework should support this.

Dependency management

If some modules depend on other modules or system dependencies, that needs to be expressed and handled by the tools. This is mainly a bake build issue, but it probably requires also some documentation in the app store that the module depends, for example, on a system library such as Boost.

Module grouping

We will probably want support for groups of modules. Users should be able to specify, for example, "Ad hoc routing" and fetch modules for several protocols, which may be hosted at different sites.