Difference between revisions of "GSOC2020AppStore"

From Nsnam
Jump to: navigation, search
(Phase 2)
(Week 9: 27th July - 2nd August)
 
(2 intermediate revisions by the same user not shown)
Line 47: Line 47:
 
* Util functions required to interact with Jenkins are added.
 
* Util functions required to interact with Jenkins are added.
 
* The code is documented.
 
* The code is documented.
 +
* Doc related to this phase - [https://docs.google.com/document/d/1KofTOsiA0I_HQFSQ_zc-iK2gdN2QRyYFoRiuG2GkIh4/edit?usp=sharing link]
  
 
===Phase 3===
 
===Phase 3===
Line 98: Line 99:
 
* Added build history with pagination
 
* Added build history with pagination
 
* Jenkins env variables loaded from settings/base.py and imported in util/build_pipeline.py
 
* Jenkins env variables loaded from settings/base.py and imported in util/build_pipeline.py
* Code formating and comment/docstings for functions
+
* Code formating and comment/docstrings for functions
 
* Added exception handling where it was needed
 
* Added exception handling where it was needed
  
 
=== Week 9: 27th July - 2nd August ===
 
=== Week 9: 27th July - 2nd August ===
* to be added
+
* Made changes to the pull request according to the discussions on the pull request.
 +
* Made Release model enforce the combined unique constraint on its app and version.
 +
* Updated build history url to more human readable /build/history/app_name/release_version/.
  
 
=== Week 10: 3rd August - 9th August ===
 
=== Week 10: 3rd August - 9th August ===

Latest revision as of 19:03, 2 August 2020

Main Page - Current Development - Developer FAQ - Tools - Related Projects - Project Ideas - Summer Projects

Installation - Troubleshooting - User FAQ - HOWTOs - Samples - Models - Education - Contributed Code - Papers

Back to GSoC 2020 projects

Project Overview

  • Project Name: App Store improvements
  • Student: Shivamani Patil
  • Mentors: Abhijith Anilkumar, Ankit Deepak, Mishal Shah, Tom Henderson
  • Project Goals: This project aims to develop a Jenkins automation server and add necessary updates to ns-3 AppStore to check if available, uploaded or updated apps/modules to AppStore build and pass tests successfully for the given ns-3 versions and display that information on AppStore. It also aims to make App Store aware of new ns-3 releases for calling build/test on all available App Store modules from Jenkins server, in turn, displaying build/test status for that ns-3 release on App Store. Ideally, it is about making App Store aware and capable of interacting with the newly developed Jenkins server.
  • Repository: https://gitlab.com/shivamanipatil/ns-3-AppStore
  • About Me: I am a second-year Information Technology undergraduate student from National Institute of Technology Karnataka(NITK), India. I am interested in computer systems, operating systems, and backend development. I have been working with Django and python for the past 1 year. I have previously worked on App Store for migrating Elasticsearch to the latest version.

Milestones and Deliverables

The whole GSoC period is divided into 3 phases. The deliverables of each phase are mentioned below:

Phase 1

  • Adding Build model to AppStore, writing tests for it and its documentation
  • Jenkins local installation
  • Building pipelines in Jenkins

Phase 2

  • Build status window on App page
  • REST API creation in AppStore, adding tests and its documentation
  • Build trigger scripts/functions

Phase 3

  • Gitlab webhooks for new ns releases
  • Deploying Jenkins on a dedicated server
  • Comprehensive App user and developer guide


Phase reports

Phase 1

  • Adding Build model to AppStore and it's test.
  • Jenkins local installation and Building pipelines in Jenkins.
  • Merge request for phase 1 work - link
  • Doc for pipelines workflow and code explanations - link
  • Doc regarding limitations of current pipeline scripts and guidelines for App release if they wish to use Jenkins server for build statuses - link

Phase 2

  • App Editors and Admins can now call the build on an app release from the App Store.
  • App Editors and Admins can now cancel the build from the App Store.
  • Build history for app releases is visible.
  • Util functions required to interact with Jenkins are added.
  • The code is documented.
  • Doc related to this phase - link

Phase 3

  • to be added

Weekly Reports

Community Bonding Period

  • Database migration for ns-3 AppStore from SQLite to Mysql/Postgres(waiting for mentors to try it on deployed site).Steps followed by me are documented in this doc.
  • Participated in the coding sprint and have provided 3 merge requests for the ns-3 app store.
  • Reviewed dependencies(doc) and provided 2 merge requests for package replacements - python-dotenv, django-admin-bootstrapped.

Week 1: 1st June - 7th June

  • The Build model to store information about the build info for an app/module was created.
  • Tests for the Build model were also added.
  • Commit for the work : link

Week 2: 8th June - 14th June

  • Couldn't work due to examinations and submissions(had pre-notified to mentors).

Week 3: 15th June - 21st June

  • Installed Jenkins on the local machine.
  • Read Jenkins documentation about pipeline building and its terminologies.
  • Explored bake for fetching and building ns-3/modules/forks.
  • Set up my local environment for pipeline building.

Week 4: 22nd June - 28th June

  • Added scripts that will be used by modules/forks with bakefile.
  • Added scripts that will be used by modules/forks providing repo link/ direct download link.

Week 5: 2th July - 5th July

  • Planned and discussed with mentors about Build triggers, who can call build on releases and overall workflow for App Store side of pipeline triggering using Jenkins APIs.

Week 6: 6th July - 12th July

  • Added functions in the App Store to call the Jenkins builds on app releases.
  • Added some guideline checks in the App Store itself so that unnecessary builds are not triggered which would surely fail due to pipeline requirements:-
   1. Check for git repo link structure - if not in standard form modify it. 
   2. Check if tag/version used for that repo exists in the git repo.
   3. Check if direct link response content-type can actually be extracted/uncompressed in the pipeline.
  • Added pipeline script support for more direct link download types zip, gzip, tar, tar.gz and tar.bz2 (first it was just tar.gz).

Week 7: 13th July - 19th July

  • Added building and aborted status types and Jenkins job URL field to the Build model.
  • Added token-based PATCH REST API in the App Store to update the Build status, Build id and Build URL from Jenkins pipelines.
  • Updated the Pipeline codes to make REST API calls to the App Store to update the Build URL and build status in Post-build section (Updating build status according to build outcome i.e aborted, success or failure.)
  • Display the latest build - (Build status, Build URL) for a release on the app page temporarily.
  • Added an additional check in the App Store to check if repo exists of module/fork first before calling 'without bakefile' pipeline builds.

Week 8: 20th July - 26th July

  • Added cancel build option
  • Added tests for PATCH REST API used by Jenkins
  • Added build history with pagination
  • Jenkins env variables loaded from settings/base.py and imported in util/build_pipeline.py
  • Code formating and comment/docstrings for functions
  • Added exception handling where it was needed

Week 9: 27th July - 2nd August

  • Made changes to the pull request according to the discussions on the pull request.
  • Made Release model enforce the combined unique constraint on its app and version.
  • Updated build history url to more human readable /build/history/app_name/release_version/.

Week 10: 3rd August - 9th August

  • to be added

Week 11: 10th August - 16th August

  • to be added

Week 12: 17th August - 23rd August

  • to be added