Last week I posted some preliminary numbers from the Google Summer of Code midterm mentor survey, and since then I have continued to crunch more numbers from the survey, beginning with an analysis of which factors are correlated with student success. To accomplish this I separated the survey into five groups, based on the student's status as reported by the mentor: Already Completed, Ahead of Schedule, On Schedule, Behind Schedule, and Far Behind Schedule. Once the answers, each representing a student, were separated out into these groupings, I compared the percentages of each group's answers.
The first question on the midterm mentor survey was, "At what point did you first make contact with your student?" The answers revealed a clear trend, showing that the earlier that students and mentors came into contact, the more likely they were doing well in the program.
(click on graph to enlarge)This comes as no surprise, as making contact with a mentoring organization early reflects a student's motivation and initiative. This comment has been heard frequently when mentors discuss hallmarks of their most successful students.
The second question on the mentor survey had to do with the frequency of mentor-student interaction. The mentors' answers to "How often do you and your student interact?" were just as revealing as the first survey question, as it showed students that have interaction with their mentors more than once a week were more likely to be on schedule, ahead of schedule, or have already completed their projects.
(click on graph to enlarge)A new area of feedback gained from this survey is how the method of communication between mentor and student correlates with the project's status. Question #4 on the midterm mentor survey was, "Of the different communication methods you use with your student, which do you use most frequently?" There were several possible answers, with IRC/instant messaging and private emails making up the majority of responses. Interestingly enough, when looking at the percentages of responses by status, a clear trend emerged here as well. Those using IRC/instant messaging as their primary method of communication were far more likely to be on or ahead of schedule than those using email.
(click on graph to enlarge)I suspect that this is because of the immediate, real time nature of IRC/instant messaging. Students and mentors may feel a greater sense of accountability when there is no time gap between responses, or perhaps since IRC/instant messaging requires coordinating a particular time to meet, there is a greater amount of commitment involved, in turn indicative of a generally higher level of commitment to the work.
One of the questions that gave a murkier picture was "How much time have you spent per week interacting with your student, on average?"
(click on graph to enlarge)I have yet to find a definite message from this graph after sorting the data several different ways. Perhaps these numbers simply tell us that the amount of time interacting is less important than the frequency of interactions.
These four graphs boil down to the following:
- The earlier a student begins interacting with the mentor/mentoring organization, the more likely the project is to be on or ahead of schedule.
- The higher the frequency of interaction, the more likely the project is to be on or ahead of schedule.
- Projects that are on or ahead of schedule are more likely to be interacting via real time methods of communication (such as IRC).
- The amount of time spend during these interactions has a less clear relationship to the project status.
By Ellen Ko, Open Source Team
