Difference between revisions of "GSOC2019AppStore"

From Nsnam
Jump to: navigation, search
 
(9 intermediate revisions by the same user not shown)
Line 14: Line 14:
  
 
* '''Code:''' [https://github.com/mishal23/ns-3-AppStore ns-3-AppStore], [https://gitlab.com/mishal23/bake/ Bake]
 
* '''Code:''' [https://github.com/mishal23/ns-3-AppStore ns-3-AppStore], [https://gitlab.com/mishal23/bake/ Bake]
 +
 +
* '''Blog:''' [https://mishal23.github.io website]
  
 
* '''Documentation:''' to be added
 
* '''Documentation:''' to be added
Line 109: Line 111:
 
* Made a Moderation Panel for the Admin, so that the users signed-in using OAuth can be moderated, after which they can have privileges as chosen by the admin. Link to PR: [https://github.com/abhijithanilkumar/ns-3-AppStore/pull/50 link]
 
* Made a Moderation Panel for the Admin, so that the users signed-in using OAuth can be moderated, after which they can have privileges as chosen by the admin. Link to PR: [https://github.com/abhijithanilkumar/ns-3-AppStore/pull/50 link]
 
* Added download stats for each app on the AppStore, whenever bake.py getconf/install is called, the download count will be increased, detailed download count per release and per date is also added. Link to PR: [https://github.com/abhijithanilkumar/ns-3-AppStore/pull/52 link]
 
* Added download stats for each app on the AppStore, whenever bake.py getconf/install is called, the download count will be increased, detailed download count per release and per date is also added. Link to PR: [https://github.com/abhijithanilkumar/ns-3-AppStore/pull/52 link]
 +
 +
=== Week 8: 17th July - 21st July ===
 +
* Updated the Search API to POST Request to take the input parameters of the configured ns version and the keyword to search on. Link to PR: [https://github.com/abhijithanilkumar/ns-3-AppStore/pull/53 link]
 +
* Updated the Bake search functionality code to get the currently enabled ns versions, and thus list the apps compatible with the version configured. The code has been updated in the PR: [https://gitlab.com/nsnam/bake/merge_requests/2 link]
 +
* Output
 +
<pre>
 +
$ bake.py search gym
 +
No apps found
 +
 +
$ bake.py configure -e ns-3.29 -e ns-3.27
 +
$ bake.py search gym                   
 +
ns3-gym (0.1) - The Playground for Reinforcement Learning in Networking Research
 +
ns3-gym (0.2) - The Playground for Reinforcement Learning in Networking Research
 +
</pre>
 +
 +
=== Week 9: 22nd July - 29th July ===
 +
* Updated the Install API to POST Request to take the input parameters of the configured ns version and the module name and version. Link to PR: [https://github.com/abhijithanilkumar/ns-3-AppStore/pull/53 link]
 +
* Updated the Bake getconf functionality code to get the currently enabled ns versions, and download the module only if the ns version is compatible with the module version
 +
* Tried to setup commento locally with nginx, wasn't successful, will try with docker.
 +
 +
=== Week 10: 30th July - 6th August ===
 +
* Tried installing commento with Docker, it's facing issue to connect to postgres database used by the commento.
 +
 +
=== Week 11: 7th August - 12th August ===
 +
* Dependency Management showing the dependencies for the app from the bakefile of the latest version, shows both the optional and the compulsory dependencies. Link to PR [https://github.com/abhijithanilkumar/ns-3-AppStore/pull/55 link]
 +
 +
=== Week 12: 13th August - 19th August ===
 +
*  Fixed the rating UI with a better UI and more features like anonymous ratings, no re-rating. Link to PR : [https://github.com/abhijithanilkumar/ns-3-AppStore/pull/56 link]
 +
* Discussed with the mentors about possible improvements to the dependency management, they suggested the improvements in UI and to keep it look separated from the main installation instructions.
 +
* Discussed with the mentors for the improvements for the rating UI, suggestions on showing the average count, and the number of ratings.
 +
 +
=== Week 13: 20th August - 25th August ===
 +
*  Improved the Dependency Management UI as per the suggestions from the mentors. Code updated [https://github.com/abhijithanilkumar/ns-3-AppStore/pull/55 here]
 +
* Fixed the issues in the existing GCI tests, and added tests for the Django REST APIs made. Link to PR [https://github.com/abhijithanilkumar/ns-3-AppStore/pull/58 here]
 +
* Improved the Rating UI as suggested by the mentors. Code updated in the PR: [https://github.com/abhijithanilkumar/ns-3-AppStore/pull/59 link]
 +
 +
Final Report: [https://mishal23.github.io/gsoc19-report/ link]

Latest revision as of 20:16, 25 August 2019

Main Page - Current Development - Developer FAQ - Tools - Related Projects - Project Ideas - Summer Projects

Installation - Troubleshooting - User FAQ - HOWTOs - Samples - Models - Education - Contributed Code - Papers

Return to GSoC 2019 Projects page.

Project overview

  • Project name: Improving the ns-3 AppStore and linking with bake
  • Abstract: The goal of the project is to link the existing app store with the bake build system and improve the AppStore. The project aims to support command line installation and update of modules. Once this project is complete, ns-3 users will be able to install a module on AppStore from the command line.
  • Documentation: to be added
  • About Me: I am a 3rd-year Computer Science undergraduate student from National Institute of Technology Karnataka (NITK), Surathkal, India. I have worked to implement TCP New Vegas in ns-3. I have about 1.5 years of experience in Python and Django.

Technical Approach

The App Store is built using the Django framework. The primary aim is to link the AppStore with bake which would require setting up REST API to the existing codebase for install/update/latest version of the modules, django-rest-framework will be used for the same. It was found out that the AppStore is built on Python2.7 and since the support is going to end in 2020, the first step of the project will be to port the AppStore codebase to the latest version of Python and Django. The REST APIs made would be used to add the command line support of install and update to bake. Later to enhance the features of the AppStore, GitHub and GitLab OAuth will be added, download statistics for each of the modules will be added. Later to support Multi-Threaded Comments, disqus will be used and the rating UI will be revamped using the django-star-ratings library. A detailed description of the project is available here

Milestones and Deliverables

The entire GSoC period is divided into 3 phases. The deliverable at the end of each phase is as mentioned below:

Phase 1

  • Refactoring the existing code and port existing code to Python 3 and the latest version of Django
  • REST APIs for install and latest version of the extension for the bake build system
  • Add install option to the bake build system

Phase 2

  • Add update option to the bake build system
  • Add GitLab/Github OAuth support to AppStore
  • Statistics (download count) for each the extension

Phase 3

  • Multi-Threaded Comment using Disqus
  • Dependency Management
  • Ratings for the extensions on the AppStore

Weekly Reports

Community Bonding Period

  • Prepared wiki page for the project.
  • Setup Bake and got familiar with the ns-3-AppStore and the Bake functioning
  • Communicated with the mentors regarding the proposal.
  • Started with the documentation for the AppStore: user-workflow, user-guide
  • Prepared a gif to include in AppStore documentation instead of images: view

Week 1: 27th May - 2nd June

  • The code for the ns-3-AppStore has been migrated to Django 2.0 and Python3.6.
  • The urlpatterns are changed to use path instead of url, since url is likely to be deprecated in future source and path uses route without regex, so simpler urls..
  • The ns-3-AppStore code is changed to follow the PEP8 guidelines.

Merge Request for the work: here

Week 2: 3rd June - 9th June

  • Built the REST APIs to support install and search bake command line options.
  • To avoid any possible attack, rate limiting for the REST APIs is enabled.
  • Merge Request for the work: here

Week 3: 10th June - 16th June

  • Documented the REST APIs created in the last week here
  • Implemented the search command line option which shows the name of the module, version and the abstract as shown below.
  • Started with the install command line option, currently it works for forked modules.

Output:

$ bake.py search gym
ns3-gym (0.2) - The Playground for Reinforcement Learning in Networking Research

Week 4: 17th June - 23th June

  • Install command line option in bake is added and works for forked as well as contributed modules.
  • One can even install a specific version of a module by specifying ==version in the command line option. (bake.py install mmwave==1.2)
  • Fetch command line option is added, which only downloads the bakefile for the specified/latest version of the module. (bake.py fetch epidemic-routing)
  • Worked on the code review for the REST APIs.
  • The entire design for the REST APIs has been revamped to follow the use of Serializers (more robust, less error prone!), Search API changed from using function-based API to ViewSet
  • An usage example of the install option is as follows:
  • Workflow document: here
  • Merge Request for the work: here

Output:

$ bake.py install epidemic-routing
Collecting epidemic-routing
Disable all non-mandatory dependencies? (Y/n): y
Downloading, building and installing the selected modules and dependencies.
Please, be patient, this may take a while!
 >> Downloading ns-3.29 - OK
 >> Downloading epidemic-routing (target directory:ns-3.29/contrib/epidemic-routing) - OK
 >> Building ns-3.29 - 

Week 5: 24th June - 30th June

  • Worked on the comments received on the code review for the bake command line capabilities
  • On discussion with the mentors, they suggested having a separate class for Bake to interact with the ns-3-AppStore, and hence a new class BaseClient was made which would handle all the necessary interactions with the ns-3-AppStore and would return only the success response in the main function of the Bake.py.
  • Based on the review, added regression tests for the command line capabilities added in Bake.
  • Made a proposal on the various Commenting Feature for the ns-3-AppStore. The proposal can be found here

Week 6: 1st July - 8th July

  • Modified the Search API to use pagination, currently the page size is set to 20. The PR is merged, link to PR: link
  • Filters on the AppStore were added to search for Apps based on the type and the category.
  • On review with the mentors, they suggested to have a minimal UI rather than a button to query and the select options, and hence the filtering technique is changed to follow one-click filter for Fork/Modules/All. Link to PR: link

Week 7: 9th July - 16th July

  • Added Github & Gitlab OAuth support to the ns-3-AppStore
  • Made a Moderation Panel for the Admin, so that the users signed-in using OAuth can be moderated, after which they can have privileges as chosen by the admin. Link to PR: link
  • Added download stats for each app on the AppStore, whenever bake.py getconf/install is called, the download count will be increased, detailed download count per release and per date is also added. Link to PR: link

Week 8: 17th July - 21st July

  • Updated the Search API to POST Request to take the input parameters of the configured ns version and the keyword to search on. Link to PR: link
  • Updated the Bake search functionality code to get the currently enabled ns versions, and thus list the apps compatible with the version configured. The code has been updated in the PR: link
  • Output
$ bake.py search gym
No apps found

$ bake.py configure -e ns-3.29 -e ns-3.27
$ bake.py search gym                     
ns3-gym (0.1) - The Playground for Reinforcement Learning in Networking Research
ns3-gym (0.2) - The Playground for Reinforcement Learning in Networking Research

Week 9: 22nd July - 29th July

  • Updated the Install API to POST Request to take the input parameters of the configured ns version and the module name and version. Link to PR: link
  • Updated the Bake getconf functionality code to get the currently enabled ns versions, and download the module only if the ns version is compatible with the module version
  • Tried to setup commento locally with nginx, wasn't successful, will try with docker.

Week 10: 30th July - 6th August

  • Tried installing commento with Docker, it's facing issue to connect to postgres database used by the commento.

Week 11: 7th August - 12th August

  • Dependency Management showing the dependencies for the app from the bakefile of the latest version, shows both the optional and the compulsory dependencies. Link to PR link

Week 12: 13th August - 19th August

  • Fixed the rating UI with a better UI and more features like anonymous ratings, no re-rating. Link to PR : link
  • Discussed with the mentors about possible improvements to the dependency management, they suggested the improvements in UI and to keep it look separated from the main installation instructions.
  • Discussed with the mentors for the improvements for the rating UI, suggestions on showing the average count, and the number of ratings.

Week 13: 20th August - 25th August

  • Improved the Dependency Management UI as per the suggestions from the mentors. Code updated here
  • Fixed the issues in the existing GCI tests, and added tests for the Django REST APIs made. Link to PR here
  • Improved the Rating UI as suggested by the mentors. Code updated in the PR: link

Final Report: link