From Nsnam
Jump to: navigation, search

ns-3 Google Summer of Code Selection Process

This page describes how the ns-3 organisation reviews student applications for the Google Summer of Code programme and narrows down on the final selections.

In summary, we will form a selection committee of mentors and maintainers who will review the submitted applications. In cases where we need to probe further for a candidate's technical skills, we will organise IRC interviews with the candidates. After the reviews and the interviews are completed, the committee will agree upon a final ranking of applications. The admins will then recommend the final ranked list of applications to Google.

Selection Committee

In 2018, the committee consists of:

  • Tom Henderson (org admin)
  • Tommaso Pecorella (org admin)
  • Natale Patriciello
  • Dizhi Zhou
  • Mohit Tahiliani
  • Matthieu Coudron

The selection committee will have access to all applications and be able to leave public and private comments, and to score the applications.

Review Committee Checklist

  1. Sign in to the Google Summer of Code site as a mentor, and pick the ns-3 organisation(the organisation name is "The ns-3 network simulator"). You won't be able to see the applications otherwise.
  2. If you are the concerned mentor for a particular application, please select "Wish to mentor" on the left bar for the application so that we can assign you the application.
  3. Review *every* application and assign a score to each one. Please see below for the scoring guidelines.
  4. After you've reviewed all applications, leave behind a *private* comment on your top 3 proposal picks, indicating your reasons for doing so.
  5. If you're trying to leave a review for other reviewers to see, please make sure *you don't* leave a public comment.

Scoring System

On Melange, every reviewer should assign a score to every application. The scoring system is as follows

  • 1  : Spam application. **For mentors of the respective projects and admins ONLY**
  • 2  : Ignore proposal. **For mentors of the respective projects and admins ONLY**
  • 3  : One of the worst proposals, other reviewers should ignore it
  • 4  : This definitely doesn't deserve to get into GSoC
  • 5  : I'd rather this didn't get a slot
  • 6* : Don't care one way or the other whether it's funded
  • 7  : Mild enthusiasm. Fairly cool but niche, or average proposal
  • 8  : Student seems quite good, and the project is well thought out
  • 9  : This is one of my favourites
  • 10 : Highly recommended. **For mentors of the respective projects and admins ONLY**
  • 11 : Strongly recommended. **For mentors of the respective projects and admins ONLY***

6 should be your default score

What are we looking for in a good proposal?

Our objective is to select students who are potential future maintainers of the ns-3 project. Over the summer, the community will be investing time and effort in mentoring the students to achieve this end. That said, this is what a good proposal will look like:

  1. A detailed proposal that sets a realistic list of deliverables, demonstrates a clear path towards achieving the same, is complete, and technically sound.
  2. The proposal is an original document authored by the candidate, and is not a copy paste job.
  3. The proposal is scoped such that it has a strong chance of being successfully completed and merged by the end of the coding period.
  4. The candidate is technically very strong. This is judged through prior experience with contributing to ns-3 and/or other open-source projects, code samples, their past experience and through interviews conducted during the review process by the mentoring team.
  5. The candidate does not have any other commitments over the summer, and can devote 40 hours per-week as is required of GSOC candidates.
  6. The proposal has been discussed at length on the mailing list and/or IRC.
  7. The student has demonstrated some interest in involvement with ns-3 either before or after GSOC ends.

Signs of a bad proposal

  1. The proposal is a regurgitation of the project description and the recommended reading.
  2. Poor professional conduct in developing the application (extensive copy-pasting for instance).
  3. There is little evidence for the candidate's technical abilities.
  4. Very little mailing list and community engagement in developing the proposal.
  5. The candidate has other commitments for the summer (vacations, exams, coursework, another job etc.).
  6. Previous failures within GSOC.

Review Process

Pre-evaluation phase

The organisation admin makes a pass through all the proposals. The admin will assign a score of 1 to spam, very weak applications, applications that do not fulfill the patch requirement and applications that are copy-paste jobs. These applications will not be considered for further review.

Timeframe: March 27-April 23, 2018

First review phase

  1. All members of the selection committee will assign scores to the applications based on the scoring system mentioned above. Note, the scores of 1, 2, 11, and 12 are reserved for the admins and the mentors of the corresponding projects.
  2. All reviewers must post a private comment for each of their top 3 proposal picks explaining their score. They are free to post comments for other applications as well.

At the end of the first review phase, admins will make a pass through the applications to ensure that scores are assigned equally across all proposals. This is for cases where some mentors are more lenient with awarding scores than others, and also to avoid situations of vested interests. Admins will adjust scores accordingly to balance against such situations.

Second review phase

Admins will present a list of second round candidates for discussion by the committee. The committee may organise interviews with the candidates to probe for technical skill and to clarify aspects of the proposal.

Final phase

Outcome of second review phase is discussed, and the final ranking list of candidates is decided. The organisation admin will then lock the final set of candidates and make the recommendation to Google.

Feedback on applications

Students who do not get selected are welcome to contact the organisation admins to solicit feedback on their applications.

Confidentiality and professional responsibilities of selection process

With respect to confidentiality and professional responsibilities of the review process, the selection committee is expected to treat the process similar to a peer reviewed, technical program committee. Example guidelines are posted here.