Wikimedia Engineering/Report/2012/April

Engineering metrics in April:

67 unique committers contributed code to MediaWiki.
The total number of unreviewed commits went from about 100 to 138.
About 49 shell requests were processed.
63 developers got developer access to Git and Wikimedia Labs, among which 60 are volunteers.
Wikimedia Labs now hosts 81 projects, 136 instances and 305 users.

Major news in April includes:

Substantial work on Wikimedia engineering's goals for the next fiscal year;
The selection of 9 Google Summer of Code students and the start of their work;
The shift to a rapid deployment cycle;
A new mobile skin deployed to Wikimedia sites;
The Wikipedia mobile app for iOS switching to using OpenStreetMap data.

Events

Recent events

OpenStack Design Summit and Conference — The Wikimedia Labs team attended this San Francisco event, collaborated on upcoming OpenStack design decisions, spoke to other users, and publicized the Wikimedia Labs project.

Upcoming events

Berlin hackathon (1–3 June 2012, Berlin, Germany) — The 120 tickets available for this three-day "inreach" hackathon disappeared as the event sold out in April. The Wikimedia technical community, including MediaWiki developers, Toolserver users, bot writers and maintainers, Gadget creators, and other Wikimedia technologists, showed substantial interest in the hackathon. The event, hosted by Wikimedia Deutschland, will mostly involve focused sprints, bugbashing, and other coding, with a few focused tutorials and trainings on Git, Lua, Gadgets changes, security, and performance optimization. Wikimedia Deutschland will also use this event to consult on and discuss the Wikidata structured data project.

Wikimania hackathon (10–11 July 2012, Washington, D.C., USA) — Katie Filbert, Gregory Varnum, and Sumana Harihareswara are planning the hybrid inreach/outreach hackathon occurring just prior to Wikimania. Experienced Wikimedia technologists will collaborate, while interested new developers will be able to learn introductory MediaWiki development. Accessibility will be one of the event themes.

Personnel

Work with us

Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.

Announcements

Matthias Mullie joined the Features team (announcement).
Faidon Liambotis joined the Operations team to work on Wikimedia Labs (announcement).
Chris Steipp joined the Platform team as Senior Security Engineer (announcement).
Tauhida Parveen joined the Platform team to work on QA and testing (announcement).
Sumana Harihareswara was promoted to Engineering Community Manager (announcement).

Operations

Site infrastructure

Servers — We added additional servers to bits.wikimedia.org in our Virginia data center for network capacity growth and redundancy. We also deployed a new udp2log server (in EQIAD) thus providing added extra capacity to collect new data for the Analytics team.

Search — After months of preparation and refactoring work with our dated Lucene implementation at the Tampa data center, we are glad to report that Peter Youngmeister (with help from Asher Feldman, Robert Stojnic and Jeff Green) successfully built and deployed the new Search infrastructure at our EQIAD data center. The performance improvement is quite amazing, at the 99th percentile level, search latency dropped from a high of 9 seconds to 1 second, and the average search is only 100ms, down from 700ms. In addition, the new infrastructure addresses some of the previous single point of failures and capacity limitations.

Databases — Asher Feldman completed the database migration/upgrade to support SHA-1 hashes in the coming MediaWiki release.

Caching — Varnish is now used in EQIAD to serve all upload (images & media) traffic (other than Europe, which has its own servers). Mark Bergsma implemented Varnish to replace our Squid instances running in Tampa. In addition to having consistent hashing, Mark ran half of the EQIAD Varnish instances with the experimental persistent storage back-end. Unfortunately, after a few days, he found showstopper bugs and reverted it to the stable version.

Networking — A secondary network transit link has been added to our EQIAD network, providing us redundancy and capacity; it comes with IPV6 enabled.

Media Storage [?] — April saw two areas of progress: the MediaWiki code to allow original media storage in Swift was deployed to production (though it is not yet in use), and further investigation into old corrupted objects continued with new evidence and cleanup. During May, we hope to begin the data migration from the older storage system into Swift, and to deploy improved monitoring and metrics.

Testing environment

Wikimedia Labs [?] — A new version of OpenStackManager was released, adding project filters for all interfaces, usability fixes and a number of bug fixes. OpenStackManager and LdapAuthentication were switched to Git, allowing a few more changes to be pushed thanks to being able to keep a stable master branch. Notable changes were per-project sudo management, allowing sysadmins in a project to manage who gets which sudo permissions in a fine-grained manner for their projects, and a change in how groups are added to LDAP for projects. A compute node (virt5) was added to the compute cluster, allowing for another 40 instances of capacity. We had an outage towards the end of the month, again due to glusterfs. We will start looking for alternatives to glusterfs soon. Sara Smollett added per-project ganglia monitoring, displaying resource graphs for instances in projects. Andrew Bogott finished work on a plugin framework for OpenStack Nova, and has added an example plugin for a SharedFS driver, which would allow us to manage gluster volumes via an API.

Backups and data archives

Data Dumps [?] — The gluster share with the last 5 or so good dumps for all projects is ready for use by Wikimedia Labs projects. A first copy of uploaded media, accessible via rsync, was announced, and some work was done on tine infrastructure to generate downloadable bundles of media per project. We're working with the Internet Archive to produce media bundles that they can host for download as well. A new version of the dump scripts was deployed with some minor bug fixes. Christian Aistleitner wrapped up work on the PHPUnit tests for the dump maintenance scripts, and discovered a problem with the database schema, which we will need to discuss with the user community in order to find a resolution that works for everyone.

Other news

There was a short site incident at our Amsterdam site on April 26th at around 6:00 (UTC), which lasted for 30 minutes and impacted some of our European users. We experienced an unusual traffic surge that overwhelmed some resources. That was quickly addressed once we found the cause.

Features Engineering

Editing tools

Visual editor — Roan Kattouw and Trevor Parscal are rewriting the underlying data model (ve.dm) to achieve feature compatibility with the parser and correct a variety of problems that have been previously deferred. Inez Korczynski and Christian Williams have been continuing their work to stabilize and integrate the content editable layer (ve.ce) and have been working with Rob Moen, who has focused on getting the user interface elements working with the content editable layer (ve.ui). Gabriel Wicke has been working on improving the parser's ability to parse pages more quickly as well as increasing compatibility with existing features such as thumbnails. A big template-heavy page like Barack Obama can now be expanded in similar time as the production parser (80 seconds on a puny laptop) and 340MB of memory. It previously ran out of memory after consuming 1.6G and running for ~30 minutes.

Editor engagement

Article feedback — Fabrice Florin worked with OmniTi to develop a range of new features for version 5 of the Article Feedback Tool (AFT5). This month, the team deployed the first versions of the abuse filter and the relevance filter, as well as new monitoring tools for editors, to help surface useful suggestions and reduce the noise on the feedback page where posts are listed for each article. Pau Giner started designing a new look and feel for that feedback page, as well as a final version of the feedback form, with guidance from Brandon Harris. We also finalized a set of special tools for oversighters, as well as new feedback links. Dario Taraborelli, Oliver Keyes and Aaron Halfaker collected and analyzed data on how prominent feedback links impact both volume and quality of user feedback. Roan Kattouw continued to review our code and help deploy weekly releases, and Matthias Mullie contributed new code for this project. We expect to complete feature development by the end of May, with full deployment in the summer.
Page Triage — The editor engagement team continued to develop its prototype of Page Triage, which provides an enhanced list of pages to be reviewed by community patrollers. Benny Situ, Ian Baker and Ryan Kaldari developed new features for the list view of this feature, while Brandon Harris designed a new curation bar to appear on the article pages. Oliver Keyes acted as community liaison and Fabrice Florin managed this project with Howie Fung. Current goals are to complete development of the new curation toolbar in May, and deploy an integrated release version the following month, along with the Article Creation landing system. The latest prototype is available on Wikimedia Labs; our first incremental deployment to the English Wikipedia is planned for the first week in May.
Article Creation Workflow — Ian Baker, Benny Situ, Ryan Kaldari and Brandon Harris have developed the Article Creation landing system, while focusing on deploying the New Page Triage (NPT) this month. Oliver Keyes prepared new templates for a proposed 'Create a Draft' section. Fabrice Florin managed the project and created a workflow diagram to illustrate the interdependencies between ACW and NTP. The goal is to collect and analyze usage data on ACW in May with Dario Taraborelli, in order to plan our next steps for this project, which is likely to be deployed alongside NPT in coming weeks. The current ACW prototype is available for testing on Wikimedia Labs.

MediaWiki infrastructure

ResourceLoader — The team is working on deploying ResourceLoader2's gadget manager on Wikimedia Labs. They also completed a number of tasks, and are preparing for a ResourceLoader tutorial at the June Berlin hackathon.

Feature support

Wikipedia Education Program — Jeroen De Dauw has made the project almost feature complete. User testing is now finalized, and the team is still trying to get started getting code review.

2012 Wikimedia fundraiser — The fundraising team deployed GlobalCollect recurring functionality, and started transition to using Git and Gerrit for its software development process. They wrote the burn up chart for the next fundraising round. The team is still working on adding staff.

Internationalization and Editor Engagement Experiments

Internationalization and localization tools — The team has completed the first round of UI designs for a Universal Language Selector (ULS) for desktop and mobile. UI/UX team members (Pau Giner and Arun Ganesh) are now implementing a prototype to showcase the first version of ULS. The team also added keymaps for language support to Narayam, added notification support to Translate, fixed bugs, reviewed code for localization support in MediaWiki 1.19, and discussed language support metrics.
Editor engagement experiments — Karyn Gladstone, Steven Walling, Maryana Pinchuk and Ryan Faulkner conducted the Necromancy experiment, emailing lapsed editors to encourage them to edit Wikipedia again. Work on the Template A/B testing project is wrapping up; a full report is expected in May. The E3 team will also be publishing details on each experiment on meta and the English Wikipedia. The technical specifications for each implementation will be posted on mediawiki.org. The team also began recruiting for its open positions; the first software engineer for the team will be joining mid May.

Mobile

Mobile Web

Mobile design — The final selection of section styles was supported by user experience testing. We also added switches between Mobile/Desktop view and Images on/off to the footer. These changes have now been deployed as the default view of mobile Wikipedia. We're thrilled to get great comments like "Wikipedia mobile has improved its user interface, simple but lovely", "Who loves the new wikipedia mobile format on your iPhone? I know I do", and "New Wikipedia mobile website is gorgeous. Sleek and fast". The first working prototype of the new navigation UI has been completed in rough form and sent out for feedback.
Mobile Contact US — Arthur Richards and Jon Robson finished the first beta version of the new "Contact us" form. It's been deployed to our beta infrastructure and we're collecting user feedback both in and about the tool. We'll be letting this sit for two weeks to collect feedback and then will re-assess for next steps.

Apps

Wikimedia Apps — Yuvaraj Pandian released new versions of our Android app and our first ever PhoneGap version of the iOS app. Issues were identified with iOS 4.x and we released an update to fix them. Yuvaraj also continued work on the API move branch. Brion Vibber pushed out a final build of the Wikipedia App to the BlackBerry market, and started experimenting with a Windows mobile version.
MobileFrontend/Photo upload — Development was paused in favor of the new mobile navigation. We explored how to collaborate with Wiki Loves Monuments (WLM), and we have decided that a standalone app for WLM is the best way to go. Our initial development will be for Android, and iOS pending progress. Brion Vibber created a rough demonstration of some of the required features on github.
UCOSP Spring 2012 — Wiktionary Mobile RC2 is available for download, and we're looking for beta testers for this release candidate. Team members are scattering for the summer term; some will be able to continue work on the Android app, while others are beginning to plan an iOS edition. The project will be featured at the Western Canadian Conference on Computing Education on May 4–5, 2012, in Vancouver, with team members participating in a panel for the Undergraduate Capstone Open Source Projects.

Alternate Access

Wikipedia over SMS & USSD — Patrick Reilly wrapped up work packaging Vumi and we're just waiting on the operations team to install the production system. Next, we'll invite users to test it over Jabber, while we work with our various partners to connect it to mobile phone networks. This will wrap up our initial implementation of this project.
Wikipedia Zero — Patrick Reilly and Dan Foy have continued to support our Zero partner roll-outs. We're testing in new countries and are updating the Zero software as necessary. Due to our partners' confidentiality requirements, we haven't been able to highlight the specific areas where testing is happening. Moving forward, we are going to work with our partners to be able to disclose more.
MobileFrontend/J2ME app — We've narrowed down our vendors and are having interviews with the finalists.

Core

GeoData Storage & API — Further development is still pending testing to see how well it works with our apps.
MobileFrontend — Patrick Reilly, Arthur Richards and Max Semenik continued work on moving MobileFrontend to MediaWiki core. We've updated its skinning, internals, and general architecture to make it more core-friendly. Patrick also posted a response to a recent news article about Wikipedia mobile's energy consumption.
Mobile QA — We've narrowed down our candidate list and are doing final interviews.
Mobile support in MediaWiki core — Max Semenik has rewritten MobileFrontend to use MediaWiki's skin infrastructure. We're continuing to further remove any Wikimedia-specific code from the extension in preparation for a migration to core. Future work will involve an easy way to create custom output based on the view (mobile, desktop, tablet).

Platform Engineering

MediaWiki Core

MediaWiki 1.19/Roadmap — We finished deploying MediaWiki 1.19 to all Wikimedia sites at the end of February. Trevor Parscal, Timo Tijhof and Roan Kattouw solved the last bug blocking the release of the tarball for third-party users. A final 1.19 tarball release was completed May 2.
MediaWiki 1.20/Roadmap — As of April 2012, core software deployments to Wikimedia sites are done from git (instead of Subversion) through incremental "wmf" branches. The first deployment of the 1.20 release cycle, labeled 1.20wmf1, was deployed to all Wikimedia sites this month; it notably brought new diff colors for improve readability. The 1.20wmf2 deployment cycle began on April 30. The 1.20.0 stable tarball is expected to be released in fall of 2012.
Continuous integration — Jenkins has been upgraded, providing a nicer GUI. Progress was made on implementing a universal linter for all gerrit changes, and not just those with modified PHP files. The TestSwarm connection with Jenkins has been established, and TestSwarm is now running MediaWiki's QUnit test suite again. Christian Aistleitner created a test suite for the (rewritten) MWDumper system, and the tests are monitored live on Jenkins.
Git conversion — MediaWiki core and all extensions deployed on Wikimedia sites are getting into regular development cycles. The second group of extensions were migrated on April 27. We've also branched and released 1.20wmf1 to most Wikimedia sites. The priorities for Gerrit are currently fixing our UTF-8 problem, improving Gerrit integration with IRC and e-mail, making project/user information more discoverable, and upgrading Gerrit to 2.3 in the next week or so.
Multimedia — Michael Dale and Jan Gerber are looking into transcoding bugs in TimedMediaHandler. A lot of issues on beta are hardware- or configuration-related (virtual instances are running out of resources) and more hardware will be going online soon. In the meantime, the client-side playback parts of the test plan are being run by Chris McMahon and Tauhida Parveen. Ben Hartshorne and Leslie Carr have been running a script to clean up corrupted thumbnails in Swift. Aaron Schulz has been fixing bugs, and is now focusing on adding concurrent file operation support to PHP cloudfiles (the PHP interface to Swift) and to MediaWiki's FileBackend code. This should help alleviate bug 34717.
Lua scripting — Tim Starling started the implementation of a replacement for MediaWiki markup-based templates, using the Lua scripting language, embedded via the Scribunto extension. The current roadmap aims for a deployment to Labs in May 2012, then to mediawiki.org; full deployment to Wikimedia sites is scheduled for 2013. Tim will lead tutorial sessions at the Berlin Hackathon 2012.
Code review management — The Wikimedia engineering 20% policy is the current approach of the Wikimedia Foundation to improving the code review situation. With the move to Git, we no longer have a code review backlog in trunk, but we are still facing a backlog of patches to review (in Gerrit and Bugzilla), RFCs to comment on, and extensions to review.

Quality assurance

QA and testing —

Wikimedia analytics

Analytics/Reportcard — David Schoonover and Fabian Kaelin continued work on the new Report card, whose prototype is available on Wikimedia Labs. Andrew Otto has been working with the Operations team to puppetize existing services, and to add a third server (Oxygen) to run filters; we are in the process of migrating bayes to stat1. Andre Engels, Erik Zachte and Diederik van Liere have worked on new mobile reports that will be integrated into the new report card.

Technical Liaison; Developer Relations

Bug management — Mark Hershberger used the deployment of MediaWiki 1.20wmf1 to gather volunteers for the Bug Squad. He has begun working with volunteers on IRC to coordinate bug-grooming activities.
Summer of Code 2012/management — Wikimedia engineers have chosen nine students for this year's program. For the next few weeks, until May 21st, the students and their mentors are working together to train the students in MediaWiki development, so that they'll have all the basic domain knowledge they'll need to succeed during the summer.
Wikimedia Foundation engineering project documentation — Guillaume Paumier modified the templates used for the Activity pages system to display more relevant information on team hubs like the Mobile team hub. He also wrote specifications for a tool to fix bugs and limitations of the current system.
Volunteer coordination and outreach — Sumana Harihareswara and other members of the Engineering Community Team continued to follow up on contacts, recruit new contributors to the Wikimedia tech community, and mentor new contributors. They granted developer access and planned upcoming events.
Wikimedia blog maintenance — Guillaume Paumier made minor changes to the theme, like not displaying trackbacks and pings in the comments list. He investigated plugins to support multiple authors, photo galleries and multilingual posts, and is testing Piwik as a possible solution to record metrics. Chad Horohoe created git repositories for the WP-Victor theme and the WMBlog plugin in Gerrit, and imported their history from GitHub.

Wikidata

The Wikidata project is funded and executed by Wikimedia Deutschland.

The Wikidata team has been working for a month now, and they've made good progress on the initial development. They published some initial assumptions and requirements, drafted and implemented the initial API, implemented functionality to enter and edit interwiki links, and started collecting use cases for Queries and Infoboxes. They held the first two rounds of office hours and publish current scrum items at m:Wikidata/Development.

Future

The engineering management team continues to update the Software deployments page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the engineering roadmap, listing ongoing and future Wikimedia engineering efforts.