October 2011 Coding Challenge

Group:	Wikimedia Engineering
Start:	2011-09
End:	2012-01

The October 2011 coding challenge ended on November 7, 2011, 23:59 UTC. Winners were announced in January 2012.

Background

Purpose of this document:

Describe the concept of coding challenges as an alternative to other engineering outreach efforts
Provide current status updates about ongoing or planned coding challenges
Provide postmortem summaries for past coding challenges

Rationale

Wikipedia is a world-changing project, and working (as volunteer, employee, or contractor) in the context of Wikimedia Foundation projects should be a compelling proposition for many of the smartest people on the planet. But, we still need to grow our pipeline both for hiring and for volunteering relative to our engineering opportunities.

Hiring

A traditional hiring process has served us well in building our team to-date, and we’ll need to continue investing in recruiters, internal referrals, conference outreach, broad online outreach, and other strategies to maximize its impact.

But, the traditional approach creates significant tax on HR/engineering, while arguably not producing the best possible outcome. Our interviewers still frequently spend time meeting with candidates who aren’t up to our expectations, in part because we’re often trying to make the best of a so-so candidate pool, in order to meet our hiring goals -- but then don't end up hiring anyway.

Moreover, our traditional hiring process creates biases, whether we like it or not. We’re less likely to do a deep-dive on a candidate from India than on one from the SF Bay Area, for simple practical reasons. Our job postings will primarily reach people through US job boards, etc. But it only stands to reason that some of the most brilliant and talented people are going to be overlooked.

What if we could get amazing people to demonstrate their skills against concrete challenges, and use the completed challenges as the first evaluation screen for all candidates -- rather than the CV? We’d still look at the CV, but at the end of the day, we’re looking for amazing, brilliant, dedicated people who will deliver excellent results. So why use the CV or job posting as the first screening device to begin with?

Our first job is to reframe the entire process. By focusing on a traditional hiring process, we’re also establishing all the normal expectations in candidates. “I wonder if they’re a decent employer”, “Will they pay well enough that I can send my kids to the right school”, “What are the career options”, “What’s the benefit package”.

Those are all important questions everyone will and should ask, but we want to make sure that they’re not the first questions people ask.

The first message we’re trying to send is this: Hey there, listen up! You’ve got a once-in-a-lifetime opportunity to do something amazing. You can help hundreds of millions of people get access to free knowledge. Are you good enough, smart enough, dedicated enough? Prove it!

The mindset that we want our candidates to be in when they start the process is: This is a big deal. This could be a hugely meaningful decision in my life. I could learn a ton of stuff, and do some really great work with some really smart people. I want to do this! Everything else follows from there.

All the messaging, therefore, has to be consistent with this desired outcome. We shouldn’t be talking about jobs first and foremost, but rather about the impact people can have on the world by working for us.

Volunteering

We have several existing funnels through which new people become MediaWiki volunteer developers. Most important among these is the process of getting involved in tool/gadget/extension/core development through activities in other Wikimedia projects. Gadgets and tools offer especially straightforward entry-level ways to get involved. GSOC is another important outreach activity that helps to cultivate potential future volunteers, as do the various hackathons that are organized around the world every year.

With that said, as our on-ramp for technical volunteers becomes more robust, it's important that we also focus on increasing the inflow of strong new potential volunteers. Code challenges present an opportunity to do so -- we can advertise them broadly and use them to identify and gradually on-board new, talented individuals.

It's critical to the success of this process that we take support, responsiveness and on-boarding seriously. We should not increase our inflow of new volunteers beyond our capacity to support and mentor them, and to review and deploy their code.

Leveraging our sites

In motivating some of the smartest people on the planet to get excited about working for and with Wikimedia, we need to leverage the sites they’re likely to use -- especially Wikipedia. Whether it’s by means of global banners, targeted banners, or less conspicuous links, we’ve seen time and again that people do and will notice when we ask for their attention.

Will we get challenge participants? Definitely. The Community Department ran a first “open call” focused on hiring mid-2010, which resulted in hundreds of applications (tracked via the foundation:Special:CommunityHiring special page). The difference here is a stronger focus on completion of challenges -- which could be too much of a deterrent to succeed in practice. But it’s worth trying.

Participant flow example

Here’s how the participant flow could work:

Participant sees a banner on Wikimedia project:
“Wikipedia is looking for the smartest, most dedicated engineers and product visionaries on the planet. Do you have what it takes to work for a non-profit that’s changing the world? Complete one of our October 15 weekend challenges!”
Participant clicks banner, visits landing page. Landing page contains:
- Preface (perhaps personalized). There should be an inspirational piece, and an explanatory one about the process.
- A set of challenges that the participant can choose to complete.
- Links for information about job openings, volunteer opportunities, etc., for folks who prefer more traditional approaches
Participant picks a challenge. The challenge page contains:
- Instructions about how to complete the challenge.
- Information about what typically happens after a submission (e.g. time to review).
A link to an “I’ve completed this challenge” page.
Participant completes the challenge.
Participant visits the “I’ve completed this challenge page. This page contains:
- A file upload form for any required file(s), including e.g. CV for participants interested in job openings
- Some questions about the candidate, similar to foundation:Special:CommunityHiring
Participant submits information
Participant receives email confirmation
Participant receives warm personal acknowledgment and is guided forward in the process as appropriate
- Simple expression of thanks without next steps for participants who lack sufficient qualifications
- Volunteer on-boarding for participants interested and qualified to get involved in development as volunteers
- Candidate screening and interview for participants (exclusively) interested in job openings

Prep materials developed for the October 2011 Coding Challenge