New Developers/Quarterly/2018-01

This is the second edition of the New Developers Quarterly, a report covering activities, metrics, surveys and lessons learned in the Onboarding New Developers program. This report covers October to December 2017.

Your questions and feedback are welcome in the Discussion page.

Timeline

Newcomer-focused events, programs, and activities between October–December 2017:

New developer activities in the Wikimedia movement
Participants of the WikiFemHack in Greece
Participants of the WikiFemHack in Greece
Participants attending a session by Marios Magioladitis at the WikiFemHack
Wikimedia mentors at Google Summer of Code mentor summit at Google headquarters in Sunnyvale, California
Wikimedia at ChickTech's High School Kick-off in San Francisco

October 5th How Technical Collaboration is bringing new developers into the Wikimedia movement, blogpost by Quim Gil
October 7th WikiFemHack, the first Wikimedia hackathon to encourage gender diversity in Greece, blogpost by Marios Magioladitis
October 13-15th Wikimedia mentors participated in the Google Summer of Code mentor summit at Google headquarters in Sunnyvale, California
October 19th Towards building an African Wikimedia Developer Community, blogpost by Felix Nartey and Derick Alangi
November 25th How two new Wikimedians spent their time at the Montreal Hackathon, blogpost by Melody Kramer
November 26th Wikimedia is taking part in Google Code-in 2017, mentoring students to January 17, 2018 on both coding and non-coding related tasks
December 1st Wikimedia Foundation funds six Outreachy interns for round 15, blogpost by Srishti Sethi
December 9-10th Wikimedia's Developer Relations organized a workshop around Google Code-in for students of the ChickTech organization in San Francisco (more details)

Key findings

1. Low response rates (~14%) for Wikimedia and Wikimania Hackathon follow-up emails

The follow-up emails included an invitation to participate in a post-event engagement research study and a question; if there is any way we could offer help. 14% newcomers responded to the email, and 6% participated in the research study. We would like to understand how useful this approach is, and more about newcomers who attend Wikimedia events. Thus, we would continue this strategy for a few iterations. In the next round, we plan to send emails immediately afterward to have higher response rates. The current study took us some time to prepare and was rolled out a few months after Wikimania.

2. New developers gain useful information through Wikimedia events

From a very few event participants that we chatted with through the post-event engagement research study, we learned that they got busy with their professional work after the Hackathon and were not contributing much to Wikimedia projects. But they had gathered enough information at the Hackathon that they could use if they want to start contributing again.

3. Newcomers continue to struggle in understanding the code contribution process

From the survey results of both the previous and the current report, it is apparent that new developers struggle initially in understanding the code contribution process, applying coding conventions, learning how to get around Gerrit, git-review, etc. We have been continuing to make progress on improving our related documentation for new developers. We will continue to watch for improvements in this area in the next editions.

4. Dedicated mentors time, a support channel, and self-contained tasks are some suggestions made by new developers

We have already piloted a test instance of Wikimedia Developer Support channel which is running on Discourse. We plan to monitor its use closely and also brainstorm on possible action items for other suggestions made by newcomers.

5. Completion rate for the new developers survey has improved by (~25%)

Shortening the survey, that we piloted among new developers for the previous edition of this report helped gather more responses.

6. Do new developers tend to be working professionals with over five years of experience developing software?

Similarly, in the previous report, the small sample of responses in the current one as well have shown a higher percentage of experienced professional developers than we expected. We will keep watching for these answers in future surveys.

7. Lack of an automated process to measure retained new developers who contribute to code repositories outside Gerrit

Our retention rates in general and more specifically of outreach programs and events are very low. But the data only covers those code contributions that have landed in Gerrit. We are currently not set up to measure code activity efficiently from outside Gerrit.

New developers metrics and trends

Volunteers contributing patches for review

140 volunteers contributed between October–December 2017. (Source)

QoQ: -6.0%. YoY: -14.6%.

New volunteers attracted

49 new volunteers contributed between October–December 2017. (Source)

QoQ: -15.5%. YoY: -31.0%.

New volunteers retained

Percentage of volunteers active one year (±3months) after their first contribution, out of all new volunteers attracted one year ago. (Source: Calculation on data)

QoQ: +6.2%. YoY: +44.6%.

Quarterly data (October–December 2017)

Review of changesets by new volunteers

233 changesets were contributed by new volunteers. (Source: Calculation on data^[1]. Data as of January 19th, 2018.)

Projects with most new volunteers

49 new volunteers contributed to 23 repositories. (Source: "Repos by New Authors")

Outreach events

Wikimedia Hackathon 2017 (May 2017):
- Attracted 37 new volunteers.
- Retained 0 new developers who were still active two quarters after the event (October–December 2017). (Source)
Wikimania Hackathon 2017 (August 2017):
- Attracted 47 new volunteers.
- Retained 0 new volunteer who was still active one quarter after the event (October–December 2017). (Source)

Outreach programs

Google Summer of Code 2017 (May–August 2017):
- Out of 4 new developers who used Gerrit and who completed the internship (7 new developers in total), we retained 0 developers
Outreachy Round 15 (December 2017–March 2018):
- Attracted 6 new developers

Survey analysis

Read through below for a quick summary:

We sent a survey to 36 new developers who submitted code for the first time between September–November 2017. We used the shorter version of the survey designed for the previous quarterly report to understand more clearly demographics and background, motivations, challenges and needs of our new developers.

Out of the 36 new developers to whom we reached out, 18 (50%) completed the survey.

Demographics and background information

Respondents of the survey were from United States of America (3), India (3), Germany (2), Cameroon, China, Finland, Italy, New Zealand, Pakistan and United Kingdom
A little over half the respondents said they belong to the working professionals category
67% identified themselves as male and 27% as female

Motivations

New developers indicated that they first heard about Wikimedia as a place to contributing code through a MediaWiki workshop in Cameroon, their work colleagues, Outreachy, familiarity with Wikimedia tech (such as by using extensions, Toolforge, MediaWiki.org, or by hosting a wiki, etc.).

Challenges

Some challenges that new developers face around Wikimedia infrastructure are: understanding the codebase, Gerrit, coding conventions, non-availability of projects in Github, determining who is the project owner or code reviewer, etc. Other behavioral challenges are deciding what task to work on, learning a language, lack of knowledge of MediaWiki, hesitating to ask questions, etc.
Learning resources and channels that new developers refer to when they get stuck: IRC, Google, Stack Overflow, MediaWiki pages, Phabricator, PHP manuals, API docs, asking an experienced developer, etc.

Experience contributing to Wikimedia

More than 75% of the respondents said they are satisfied with their experience contributing to Wikimedia. More than half of the respondents to this survey said they are likely to recommend Wikimedia for code contribution to friends and colleagues.

Suggestions for improvement

Here are some suggestions made by new developers (responses below have been only slightly edited):

How-to documentation
An automated way to convert files to adhere to the coding style guide
GitHub pull requests
Dedicated mentors time to help newcomers initially when they begin contributing
An IRC channel for new contributors where everyone is there to help one another
Easy steps in the task description to reproduce a problem

Actions items from the previous report

We discussed the possible actions items from the previous report and made some progress on them:

Footnotes

↑ Steps: After extracting the list of "New Authors", use that list to filter on authors on Gerrit and its "Status" section.

[1] Steps: After extracting the list of "New Authors", use that list to filter on authors on Gerrit and its "Status" section.

[1]