GSOCSelectionProcess

From Nsnam
Jump to: navigation, search

Main Page - Current Development - Developer FAQ - Tools - Related Projects - Project Ideas - Summer Projects

Installation - Troubleshooting - User FAQ - HOWTOs - Samples - Models - Education - Contributed Code - Papers

ns-3 Google Summer of Code Selection Process

This page describes how the ns-3 organisation reviews student applications for the Google Summer of Code programme and narrows down on the final selections.

In summary, we have formed a selection committee of mentors and maintainers who will review the submitted applications. In cases where we need to probe further for a candidate's technical skills, we will organise chat interviews or email exchanges with the candidates. After the reviews and the interviews are completed, the committee will agree upon a final ranking of applications. The admins will then recommend the final ranked list of applications to Google.

Selection Committee

The selection committee will have access to all applications and be able to leave public and private comments, and to score the applications.

Scoring System

In 2019, Google has changed the review system and no longer uses or tracks numerical scoring. However, they want projects to select 'outstanding' (or at the very least, 'very good') applications; how these are defined are left up to the individual projects.

ns-3 will likely be allocated at most four or five slots, and we will not know in advance how many we will receive. We will therefore need to come to consensus or executive decision on what ranking we make for the top applications. We can use the 'star' system to flag those applications that a reviewer believes should be in consideration for the top four or five. Additional details that will allow us to order proposals will need to be discussed outside of the Google tool.

What are we looking for in a good proposal?

Our objective is to select students who have convinced us that they should be able to deliver a nice project and who are potential future contributors to the ns-3 project. Over the summer, the community will be investing time and effort in mentoring the students to achieve this end. That said, this is what a good proposal will look like:

  1. A detailed proposal that sets a realistic list of deliverables, demonstrates a clear path towards achieving the same, is complete, and technically sound.
  2. The proposal is an original document authored by the candidate, and is not a copy paste job.
  3. The proposal is scoped such that it has a strong chance of being successfully completed and merged by the end of the coding period.
  4. The candidate is technically very strong. This is judged through prior experience with contributing to ns-3 and/or other open-source projects, code samples, their past experience and through interviews conducted during the review process by the mentoring team.
  5. The candidate does not have any other commitments over the summer, and can devote the time required of GSOC candidates.
  6. The proposal has been discussed at length on the mailing list and/or Zulip.
  7. The student has demonstrated some interest in involvement with ns-3 either before or after GSOC ends.

Signs of a bad proposal

  1. The proposal is a regurgitation of the project description and the recommended reading.
  2. Poor professional conduct in developing the application (extensive copy-pasting for instance).
  3. There is little evidence for the candidate's technical abilities.
  4. Very little mailing list and community engagement in developing the proposal.
  5. The candidate has other commitments for the summer (vacations, exams, coursework, another job etc.).
  6. The proposal or patch submitted for review is sloppy (in grammar, spelling, formatting, etc.)

Review Process

Pre-evaluation phase

The organisation admin makes a pass through all the proposals. The admin will mark as ignored the incomplete, out of scope, or very weak applications. These applications will not be considered for further review.

First review phase

  1. All members of the selection committee will review the remaining applications. If there is an application that the reviewer would like to mentor (if selected), the reviewer should select 'Want to mentor'. Note that more than one person can select this; we can sort out details and overlaps later.
  2. If there is an application that the reviewer thinks is high enough quality to merit consideration for a final slot, then the reviewer may leave a comment in the Internal Review form that the application should be further considered.

At the end of the first review phase, the applications drawing interest from the committee will be taken into the next review phase.

Second review phase

Committee members will review again each proposal still under consideration and, if they have not done so already, will leave a comment on the internal review tool, indicating what they believe are strengths and weaknesses of the proposal and on the relative ranking of the proposal (with respect to other proposals). If any clarifications from student applicants are needed, they can be queried at this time.

Final phase

The selection committee will form a ranking of the remaining proposals by consolidating inputs from the internal reviews, and try to achieve consensus or executive decision on the ranking and on how many slots to request from Google.

Feedback on applications

Students who do not get selected are welcome to contact the organisation admins to solicit feedback on their applications.

Confidentiality and professional responsibilities of selection process

With respect to confidentiality and professional responsibilities of the review process, the selection committee is expected to treat the process similar to a peer reviewed, technical program committee. Example guidelines are posted here.