Back to GSoC 2020 projects
- 1 Project Overview
- 2 Milestones and Deliverables
- 3 Phase reports
- 4 Weekly Reports
- 4.1 Community Bonding Period
- 4.2 Week 1: 1st June - 7th June
- 4.3 Week 2: 8th June - 14th June
- 4.4 Week 3: 15th June - 21st June
- 4.5 Week 4: 22nd June - 28th June
- 4.6 Week 5: 2th July - 5th July
- 4.7 Week 6: 6th July - 12th July
- 4.8 Week 7: 13th July - 19th July
- 4.9 Week 8: 20th July - 26th July
- 4.10 Week 9: 27th July - 2nd August
- 4.11 Week 10: 3rd August - 9th August
- 4.12 Week 11: 10th August - 16th August
- 4.13 Week 12: 17th August - 23rd August
- Project Name: App Store improvements
- Student: Shivamani Patil
- Mentors: Abhijith Anilkumar, Ankit Deepak, Mishal Shah, Tom Henderson
- Project Goals: This project aims to develop a Jenkins automation server and add necessary updates to ns-3 AppStore to check if available, uploaded or updated apps/modules to AppStore build and pass tests successfully for the given ns-3 versions and display that information on AppStore. It also aims to make App Store aware of new ns-3 releases for calling build/test on all available App Store modules from Jenkins server, in turn, displaying build/test status for that ns-3 release on App Store. Ideally, it is about making App Store aware and capable of interacting with the newly developed Jenkins server.
- Repository: https://gitlab.com/shivamanipatil/ns-3-AppStore
- Final report: https://shivamanipatil.github.io/gsoc-2020-report/
- About Me: I am a second-year Information Technology undergraduate student from National Institute of Technology Karnataka(NITK), India. I am interested in computer systems, operating systems, and backend development. I have been working with Django and python for the past 1 year. I have previously worked on App Store for migrating Elasticsearch to the latest version.
Milestones and Deliverables
The whole GSoC period is divided into 3 phases. The deliverables of each phase are mentioned below:
- Adding Build model to AppStore, writing tests for it and its documentation
- Jenkins local installation
- Building pipelines in Jenkins
- Build status window on App page
- REST API creation in AppStore, adding tests and its documentation
- Build trigger scripts/functions
- Gitlab webhooks for new ns releases
- Deploying Jenkins on a dedicated server
- Comprehensive App user and developer guide
- Adding Build model to AppStore and it's test.
- Jenkins local installation and Building pipelines in Jenkins.
- Merge request for phase 1 work - link
- Doc for pipelines workflow and code explanations - link
- Doc regarding limitations of current pipeline scripts and guidelines for App release if they wish to use Jenkins server for build statuses - link
- App Editors and Admins can now call the build on an app release from the App Store.
- App Editors and Admins can now cancel the build from the App Store.
- Build history for app releases is visible.
- Util functions required to interact with Jenkins are added.
- The code is documented.
- Doc related to this phase - link
- Installation and contributing guide for ns-3 appstore.
- Readme documentation for setting the Jenkins server.
- Adding the pipelines to deployed Jenkins server.
- Testing Jenkins pipelines with dev appstore server.
Community Bonding Period
- Database migration for ns-3 AppStore from SQLite to Mysql/Postgres(waiting for mentors to try it on deployed site).Steps followed by me are documented in this doc.
- Participated in the coding sprint and have provided 3 merge requests for the ns-3 app store.
- Reviewed dependencies(doc) and provided 2 merge requests for package replacements - python-dotenv, django-admin-bootstrapped.
Week 1: 1st June - 7th June
- The Build model to store information about the build info for an app/module was created.
- Tests for the Build model were also added.
- Commit for the work : link
Week 2: 8th June - 14th June
- Couldn't work due to examinations and submissions(had pre-notified to mentors).
Week 3: 15th June - 21st June
- Installed Jenkins on the local machine.
- Read Jenkins documentation about pipeline building and its terminologies.
- Explored bake for fetching and building ns-3/modules/forks.
- Set up my local environment for pipeline building.
Week 4: 22nd June - 28th June
- Added scripts that will be used by modules/forks with bakefile.
- Added scripts that will be used by modules/forks providing repo link/ direct download link.
Week 5: 2th July - 5th July
- Planned and discussed with mentors about Build triggers, who can call build on releases and overall workflow for App Store side of pipeline triggering using Jenkins APIs.
Week 6: 6th July - 12th July
- Added functions in the App Store to call the Jenkins builds on app releases.
- Added some guideline checks in the App Store itself so that unnecessary builds are not triggered which would surely fail due to pipeline requirements:-
1. Check for git repo link structure - if not in standard form modify it. 2. Check if tag/version used for that repo exists in the git repo. 3. Check if direct link response content-type can actually be extracted/uncompressed in the pipeline.
- Added pipeline script support for more direct link download types zip, gzip, tar, tar.gz and tar.bz2 (first it was just tar.gz).
Week 7: 13th July - 19th July
- Added building and aborted status types and Jenkins job URL field to the Build model.
- Added token-based PATCH REST API in the App Store to update the Build status, Build id and Build URL from Jenkins pipelines.
- Updated the Pipeline codes to make REST API calls to the App Store to update the Build URL and build status in Post-build section (Updating build status according to build outcome i.e aborted, success or failure.)
- Display the latest build - (Build status, Build URL) for a release on the app page temporarily.
- Added an additional check in the App Store to check if repo exists of module/fork first before calling 'without bakefile' pipeline builds.
Week 8: 20th July - 26th July
- Added cancel build option
- Added tests for PATCH REST API used by Jenkins
- Added build history with pagination
- Jenkins env variables loaded from settings/base.py and imported in util/build_pipeline.py
- Code formating and comment/docstrings for functions
- Added exception handling where it was needed
Week 9: 27th July - 2nd August
- Made changes to the pull request according to the discussions on the pull request.
- Made Release model enforce the combined unique constraint on its app and version.
- Updated build history url to more human readable /build/history/app_name/release_version/.
Week 10: 3rd August - 9th August
- Added a periodic pipeline which builds the releases using ns-3-dev.
- Added a REST API that is used by this pipeline to inform AppStore to call builds on these releases.
- Started writing rough documentation about my GSoC work.
Week 11: 10th August - 16th August
- Added a pipeline to deployed Jenkins Server.
- Tested and debugged communication between deployed Jenkins and dev ns-3 appstore server.
Week 12: 17th August - 23rd August
- Added contributing and installation guides.
- Added README doc for setting the Jenkins server as per the project.