Summer of Code/Proposal Evaluation

In order to help consistently, effectively, and efficiently evaluate GSoC student applications, specific evaluation criteria should be used by project mentors. GSoC applications are to be scored on a scale of 0 to 6 where 0 is a poor candidate that meets no evaluation criteria and 6 would indicate a perfect candidate that fulfills all criteria.

In order to score applications, each mentor needs to indicate the project(s) they would be willing to mentor by selecting the "I am willing to Mentor" button on the mentor dashboard. For scoring an application, apply the below 6 evaluation criteria and evaluate a judgement scoring between 0.0 and 1.0 for each. Round the cumulative score to the closest integer when ranking.

Evaluation Criteria

• Significance: Application Impact and Relevance
  • How many people would the proposal prospectively benefit?
  • Does it have benefits outside of our community?
• Approach: Application Quality and Feasibility
  • How well is their application written?
  • Does what they propose seem reasonable?
• Visibility: Responsive and Engaging
  • Have they been active in the channel?
  • Do they respond to questions and engage in discussions?
• Charisma: Personality and Ease-of-Interaction
  • Are they easy to talk to?
  • How well do they interact with others?
• Potential: Long Term Developer Karma
  • Are they excited about working on our project?
  • Perceived chance that they will stick around and keep coding?
• Ability: Technical Coding Aptitude
  • How impressive is their patch?
  • Can they actually write code?

Mentors should have received an invite for a Google Doc spreadsheet to help evaluate and rank applications.

Notes

The Google mentor evaluation form allows for three positive and two negative evaluation levels (in addition to unevaluated/zero): +4 +2 +1 -1 -2. The sum of all your rank selections should equal more than 0 and less than 7. To set a rank of 6, you use +4+2; for 3 you set +2+1; etc. Additionally:

• Do not use negative scores except to correct or adjust your own evaluation.
• Do not adjust or compensate for other mentor scores (up or down) even to correct an obvious mistake.
• Do not evaluate them if you don't care for their proposal and/or don't really intend to mentor that student.
• Do not base your evaluation on the evaluations and comments of others.

Scores for each application will be averaged and weighted by mentor assignments after evaluations are complete and plugged into a spreadsheet for group review before the final ordering.