Wikimedia Engineering/2013-14 Goals

FY2012-13

Wikimedia Engineering Goals, FY2013-14 (July-June)

FY2014-15

Purpose of this document: Goals for the Wikimedia Engineering and Product Development department, fiscal year 2013-14 (July 1, 2013 - June 30, 2014). Our top priorities for the year are:

Stabilization of VisualEditor and development of experimental opt-in real-time collaboration mode;
Continued focus on mobile contributions, expanding beyond mobile uploads;
Continued focus on recruiting, activating, retaining and reactivating editors through smaller interventions;
Development of Flow, an alternative to talk pages and markup-based workflows.

This document was first drafted in June-July 2013, and is being continuously updated. With the exception of milestones called out in the Annual Plan (flagged as such below), we may flexibly and continuously change quarterly objectives in response to successes and failures, as well as new opportunities.

Goals by team

Site Operations and Site Architecture

Team members:

Quarter	Goals
Jul - Sep 2013
Oct - Dec 2013	Complete turning up ULSFO datacenter Complete migration of SWIFT storage to EQIAD Build out EQIAD capacity (Row D) Migrate active databases to MariaDB Initial build of OSM tileserver for the mobile team
Jan - Mar 2014	Complete technical review and proposal for enabling HTTPS-as-default for users Decommission Tampa Datacenter Complete buildout on new Datacenter facility Capacity upgrade for the External storage cluster
Apr - Jun 2014	Move to Beta Labs as a full test/staging platform for production changes Dual-homed (active/active) datacenter implementation for supported applications and tools Refactor firewalls and security configuration across datacenters Complete work on modularizing puppet configuration MariaDB automated master/shard failover Refactor, scale, and expand monitoring infrastructure

Interdependencies:

Wikimedia Labs

Team members: Ryan Lane, Andrew Bogott, Marc-Andre Pelletier, (secondary) Sumana Harihareswara

Quarter	Goals
Jul - Sep 2013
Oct - Dec 2013	Complete migration of Wikimedia Labs to EQIAD from Tampa
Jan - Mar 2014
Apr - Jun 2014

Interdependencies:

Language Engineering

Team members: Alolita Sharma, Amir Aharoni, David Chan, Kartik Mistry, Niklas Laxström, Pau Giner, Runa Bhattacharjee, Santhosh Thottingal, Siebrand Mazeland, Sucheta Ghoshal.

See also the complete 2013-2014 Roadmap on Google Docs.

Quarter	Goals
Jul - Sep 2013	ULS usage analysis, ULS completion Complete initial Content Translation design
Oct - Dec 2013	TwnMainPage extension completion RFC MediaWiki Localisation format and implementation RFC MediaWiki Frontend localisation update
Jan - Mar 2014	MediaWiki Frontend localisation update implementation Start of work on Content Translation feature First deployment of Content Translation as Beta feature
Apr - Jun 2014	More Content Translation On-screen keyboards in jQuery.IME (if time permits)

Interdependencies: VisualEditor priorities

MediaWiki Core

Team members: Greg Grossmeier, Chad Horohoe, Brad Jorsch, Antoine Musso, Aaron Schulz, Sam Reed, Tim Starling, Chris Steipp, Bryan Davis

Quarter	Goals
Jul - Sep 2013 review	Successfully review and prioritize all Requests for Comments (RFCs) for MediaWiki (by Wikimania) Initial deployments of ElasticSearch to improve search robustness
Oct - Dec 2013 review	Incremental improvements to deployment infrastructure Incremental improvements to caching Further deployments of ElasticSearch indexing technology Architectural changes routinely reviewed via the RFC process Complete OAuth deployment
Jan - Mar 2014 review	Hold first annual architectural summit Deployment of HipHop Virtual Machine (HHVM) to Beta Cluster Further deployments of ElasticSearch indexing technology Improvements to deployment infrastructure
Apr - Jun 2014 review	Deployment of HipHop Virtual Machine (HHVM) to production cluster Architectural guidelines successfully written and adopted. Complete deployment of ElasticSearch indexing technology Improvements to SecurePoll for more robust election setup

Interdependencies:

Release and QA

MW Core team members: Greg Grossmeier, Sam Reed, Antoine Musso, Bryan Davis

QA Team members: Chris McMahon, Željko Filipin, Rummana Yasmeen (VisualEditor)

Interdependencies: MediaWiki Core, Site Operations and Site Architecture, and Wikimedia Labs

Quarter	Goals
Jul - Sep 2013 review	Align browser test coverage to high profile features. Browser tests reliably tracking features across WMF software development projects in beta cluster Start comprehensive quarterly assessments of postmortems Successfully streamline the review and deployment of extensions
Oct - Dec 2013 review	Browser tests managed in feature repos with feature teams Successfully managed the first release of MediaWiki in conjunction with our outside contractor More comprehensive quarterly assessments of postmortems
Jan - Mar 2014 review	Create process documentation for ideal test/deployment steps (eg: ThoughtWorks exercise) (January) Automated API integration tests in important areas (January/early February) Integrate failed browser tests with repo diagnostics in near-real time. (In process, started with ULS) (Feb-Apr) Deployment tooling (Feb-Apr) Process through all (useful) pain points from the Dev/Deploy review session Scap incremental improvements Have Beta Cluster emulate production db setup more closely (eg: separate slaves) (Feb-Apr)
Apr - Jun 2014 review	Deployment tooling: Integrate HHVM support into our deployment system start the scap(py) & trebuchet integration conversation Beta cluster: Complete transition to scap as code deploy system Support HHVM deployment tooling and puppet configuration testing MediaWiki Release: Successfully support the release of MediaWiki 1.23 Kickoff/complete second RFP QA: (From last quarter) Use tags to run builds appropriate to released versions (e.g. don't run master build on test2wiki) Retire Cloudbees Jenkins instance Integrate WMF Jenkins with new WMF SauceLabs account Execute tests in parallel Use API to create test data at runtime more widely (not just for MobileFrontend but also VisualEditor, Flow, local dev env etc.) Add browsertests to new repos e.g. GettingStarted Hiring: Complete hiring and train new Release Engineer and Automation Engineer (Ruby)

Engineering Community

Team members: Sumana Harihareswara, Quim Gil, Guillaume Paumier, Andre Klapper

Quarter	Goals
Jul - Sep 2013 review	Several (6) volunteers know how to contribute to and fix automated tests, and do so regularly, improving our capacity for faster deployments. To improve awareness of Wikimedia's engineering needs, we ~~systematically reach out to FLOSS projects we rely on~~ (moved to Q3), and grow Tech Ambassadors membership.
Oct - Dec 2013	Project management tools: requirements and options Google Code-in Grow Tech News Revamped EC team way of working
Jan - Mar 2014	Project management tools: plan agreed and started We innovate in Outreach Program for Women mentorship, and include new areas of engineering We use a developer/community metrics dashboard to understand what's been languishing the longest, and improve the patch process We systematically reach out to FLOSS projects we rely on^[1] Architecture process and queue of Requests for comment clean.
Apr - Jun 2014	Project management tools: new tools recommended, migration plan agreed, implementation started. The Architecture process has its own inertia, backed by revamped documentation and a Requests for comment queue clean. Our list of upstream projects is complete, with contacts appointed for each project. Google Summer of Code, FOSS Outreach Program for Women, and Facebook Open Academy focus in quality, getting code merged, and potential long term contributors. Our community metrics dashboard is used to attract contributions to the most neglected areas.

Interdependencies: Legal & Community Advocacy, Release logistics and QA, Wikimedia Labs, Analytics

Multimedia

Team members: Fabrice Florin, Mark Holmquist, Gergő Tisza (joined October 2013), Gilles Dubuc (joined January 2014), Pau Giner, Keegan Peterzell.

Some of these milestones for fiscal year 2013-2014 have been updated, based on community and team discussions. The projects we are considering next are outlined these multimedia Q3 review slides and meeting notes

Quarter	Goals
Jul - Sep 2013	Started recruiting multimedia team Specifications for viewing, notifications and feedback tools to support media contributions Implemented new Gallery Layout
Oct - Dec 2013	Developed Media Viewer for showing large images and improving the viewing experience Developed Beta Features, to help users try new features before their wide release (with VE/design teams) Review new code for the GLAM Toolset (for batch uploads)
Jan - Mar 2014	Expand Media Viewer v0.2 with better UI, faster image load, more metadata, use this file Design new Media Viewer 0.3 features (video/audio support, slides, collections, zoom) Discuss a strategy for new Video formats in RfC with community members Engage community in discussing 2016 Multimedia Vision and team planning
Apr - June 2014	Release Media Viewer (pilots, metrics, wide release, bug fixes) Improve Upload Wizard (metrics, planning, bug fixes, incremental code refactoring, UX improvements) Start File Feedback experiment (to surface useful content based on viewer feedback) Discuss plans for better search and structured data on Commons with community members and Wikidata team Develop first File Notifications (e.g. 'your file was used in an article' - if time allows)

For more information, view these multimedia Q3 review slides and meeting notes.

Interdependencies: MediaWiki Core, E2, Mobile, Design

Based on community feedback, we identified five main areas of activity 2013-2014: improving the viewing experience and contributions/upload pipeline in the first half of the year, then focusing on file curation, discovery and placement in articles for the second half of the year. Our overall goals for this year are to increase both the number of contributions and files used in Wikipedia articles. Read more on this multimedia planning page.

Editor Engagement - Core Features

Team members:

Maryana Pinchuk (PM)
Erik Bernhardson
Benny Situ
Matthias Mullie
S Page (scrummaster)
Shahyar Ghobadpour
Brandon Harris (designer)
Vibha Bamba (designer)
May Galloway (designer)
Oliver Keyes (product analyst)
Nick Wilson (community liaison)

Quarter	Goals
Jul - Sep 2013
Oct - Dec 2013	Build prototype of minimum viable product (user-to-user discussion software for a shared community discussion space) on WMF Labs. Invite WikiProject users to test the software. Deploy as opt-in beta to the WikiProjects that have agreed to trial the MVP.
Jan - Mar 2014	Work on more features per community feedback. Enable Flow on more WikiProjects and user pages of any users interested in opting in Start planning collaboration workflows for article talk space.
Apr - Jun 2014	TBD Enable by default on new user talk pages. Deploy to article talk. Start planning features/workflows for other discussion spaces

Note: Our initial annual plan commitment was: "By end of December 2013, user-to-user messaging will be fully deployed in production on one or more Wikipedia projects. This release would be available to all user groups, and deployments in additional languages and projects would follow soon after the initial release. By end of July 2014, visible progress will have been made towards Flow-enabled article talk discussions, a workflow language and feed/subscriptions framework, with precise milestones to be determined. However, we don't expect that we will have Flow on all talk-based workflows by year-end." We have revised this plan somewhat.

Interdependencies:

Editor Engagement - Growth Team

Team members:

Matt Flaschen (Software Engineer)
Aaron Halfaker (Research Scientist)
Kaity Hammerstein (Associate Interaction Designer)
Rob Moen (Software Engineer)
Sam Smith, (Software Engineer)
Moiz Syed (Interaction Designer)
Steven Walling (Product Manager)

Quarter	Goals
Jul - Sep 2013	Done Finalize the first-time editor experience in GettingStarted (including final A/B test), add a task API to GettingStarted to support a mobile Web version of GettingStarted.
Oct - Dec 2013	Done Launch onboarding (GettingStarted + GuidedTour) to at least one non-English Wikipedia. Done Refactor GuidedTour to improve the API and usability. Done Launch first experiments to convert anonymous editors and readers to registered Wikipedians.
Jan - Mar 2014	MVP Done Test improvements to Wikipedia article creation workflows with a focus on new user onboarding -- including landing pages, potential Draft namespace and more. N Not done Test task suggestions and workflows to improve retention of new Wikipedians gained through onboarding and anonymous editor acquisition campaigns.
Apr - Jun 2014	Annual Plan commitment: By end of June 2014, we expect to achieve a sustainable increase in the Total Active Editors core metric (registered users across all projects who make >= 5 contributions in content namespaces, de-duplicated) by 2.4K per month, adjusting for seasonality and length-of-month.

Learn more about how we formed our 2013-14 goals, and the details for each product area.

Interdependencies: Analytics, Design, VisualEditor, Mobile Web.

Mobile

Team members:

Mobile web:

Arthur Richards
Jon Robson
Juliusz Gonera
Ryan Kaldari
Max Semenik

Apps:

Monte Hurd
Brion Vibber
Yuvaraj Pandian

Product:

Kenan Wang

Quarter	Goals
Jul - Sep 2013	Apps: Uploading workflows for Wikipedia Commons application. Mobile web: Release editing to the full mobile site on all projects. Release notifications to projects where Echo is enabled (English Wikipedia, Mediawiki.org).
Oct - Dec 2013	Goal: On enwiki, 1000 mobile registrations become active editors (5+ ns0 edits) every month. Mobile web: On-boarding experiments: Getting started on mobile, mobile help and documentation. Experiments with user profiles and humanizing Wikipedia. Micro-contribution experiments around location and uploads in beta. Research VisualEditor on tablets. High-level goal: increase number of mobile registered users that become active editors. Apps: Release beta campaigns application for Wikipedia Commons. Begin work on Wikipedia Native apps (iOS and Android).
Jan - Mar 2014	Mobile web: VisualEditor on tablets. Experiments with power users features: article history, reversion, article patrolling, gnoming, etc. Experiments with Flow on Mobile. Apps: Finish and launch Wikipedia native apps minimum viable product (iOS and Android).
Apr - Jun 2014	Mobile web: Begin directing tablets to the mobile site by default. Promoting power user features to stable with tablet interface. Further integration of Flow on Mobile. Annual Plan commitment: By end of June 2014, the number of unique registered users per month who contribute content to the projects from a mobile device (i.e. at least 1 mainspace edit or upload) will increase to 6K.

Interdependencies:

VisualEditor

Team members:

James Forrester (PM)
Trevor Parscal (Lead)
Roan Kattouw
Ed Sanders
Rob Moen
Timo Tijhof
Kaity Hammerstein (design)

Quarter	Goals
Jul - Sep 2013	Client-side peformance improvements, increased stability, technical debt reduction and wider language support Improvements for core editor features like usability in editing references, templates & citations, and copy-and-paste support
Oct - Dec 2013	Annual Plan commitment: By end of December 2013, after additional stability improvements and bug fixes, VisualEditor will be deployed as the default editor for most Wikimedia wikis, beyond the July Wikipedia rollout. Further work on improved performance, increased stability, and reduced technical debt.
Jan - Mar 2014
Apr - Jun 2014	Annual Plan commitment: By end of June 2014, opt-in experimental real-time collaboration and chat will be deployed in production, leading to full build-out in the default mode in FY 2014-15.

Interdependencies: Depends on Parsoid for Wikitext <=> HTML conversion.

Parsoid

Team members:

Gabriel Wicke (Lead)
Subbu Sastry
C Scott Ananian
Arlo Breault (contractor)
Marc Ordinas i Llopis (contractor)

See detailed roadmap.

Quarter	Goals
Jul - Sep 2013	Compatibility improvements, cleanup, start work on variant conversion and HTML storage. Provide a public HTML API.
Oct - Dec 2013	Implement HTML storage in core, enforce nesting of transclusions, transclusion parameters as DOM, start work on HTML-only wiki support.
Jan - Mar 2014	HTML-only wiki support, non-Wikipedia project support
Apr - Jun 2014	Stretch goal: Use Parsoid HTML for all page views

Interdependencies: Depends on PHP preprocessor via API. Used by the VisualEditor and (as wikitext UI for HTML content) by Flow. Stable HTML DOM API useful for anybody working with our content using standard HTML/XML tools (Kiwix, Google, mobile, search, bots,..).

Analytics

Team members:

Development

Dan Andreescu
Christian Aistleitner
Stefan Petrea
Nuria Ruiz
Charles Salvia

Research and Data Analysis

Dario Taraborelli
Erik Zachte
Aaron Halfaker
Leila Zia

Quarter	Goals
Jul - Sep 2013	Product Goals Fix What's Broken Stabilize Hadoop Cluster Resolve Production Issues (Wikipedia Zero Dashboards, Program & Evaluation Dashboards) Support Key Initiatives Wikimetrics V2 (Program Development/Grantmaking) Metric: Cards in the Mingle Epics for the quarter scheduled and completed (Mingle burndown charts) Research Goals Support Editor Engagement program with data modeling, experimental design and data analysis Support other focus areas as needed/resourced Engineering Goals To encourage community participation, Wikimetrics has at least 95% test coverage Organizational Goals Organizational design and hiring infrastructure in place and functional (Engineering and Research) -- Metric: documents available, candidates interviewed First draft of Stakeholder engagement (Engineering and Research) and agile execution models designed, implemented and shared with foundation -- Metric: documentation available and socialized Participate in Privacy Policy Discussion (Driven by legal) First draft of Foundation's Analytics Epics -- Metric: Epics visible on Wiki
Oct - Dec 2013	Product Goals Fix What's Broken Stabilized, Hardened, and Monitored Hadoop Cluster is operational Mobile click-stream delivered into Hadoop via Kafka Support Key Initiatives Wikistats Evolution plan Additional Deliverables driven by Epic prioritization Research Goals Support other focus areas as needed/resourced Engineering Goals End to end monitoring of production services Documentation of engineering processes Organization Goals Continue to drive hiring: maintain interview cadence (1-3/week) Standardize tools sets and best practices for data modeling and QA (Research)
Jan - Mar 2014
Apr - Jun 2014

Interdependencies:

Fundraising Engineering

Team members:

Katie Horn (Lead)
Adam Wight
Matthew Walker
Sherah Smith
Jeff Green (Ops)
(Fundraising team)

Quarter	Goals
Jul - Sep 2013	Preparation for English fundraiser. Work on additional new CC payments gateway (Adyen) and campaign testing for Paypal and Amazon Payments.
Oct - Dec 2013	English fundraiser launch.
Jan - Mar 2014	International campaigns. Initial integration with WorldPay.
Apr - Jun 2014	More international campaigns.

Interdependencies:

Notes

↑ Wikimedia depends on a ton of other FLOSS projects, from Firefox to varnish to MariaDB to jQuery. We have no great list of what all those projects are and what we might want out of those upstreams (beyond what's filed in Bugzilla under the "upstream" keyword). ECT is a natural leader to start and maintain those collaborations. So we should take some time to reach out to our tech community, at WMF & externally, to start collating that list and getting some idea of what's high-priority and what's possible to push on. For instance, HipHop and varnish collaborations are pretty much ongoing and might just need a little monitoring. On the other hand, we're not really telling the GIMP or Chrome or Apache our wishlists. In the long run these collaborations may also lead to getting more volunteers in WM spaces, which would be great. But in the near term, we ought to help keep bug reports and wishes from falling through the cracks.

[1] Wikimedia depends on a ton of other FLOSS projects, from Firefox to varnish to MariaDB to jQuery. We have no great list of what all those projects are and what we might want out of those upstreams (beyond what's filed in Bugzilla under the "upstream" keyword). ECT is a natural leader to start and maintain those collaborations. So we should take some time to reach out to our tech community, at WMF & externally, to start collating that list and getting some idea of what's high-priority and what's possible to push on. For instance, HipHop and varnish collaborations are pretty much ongoing and might just need a little monitoring. On the other hand, we're not really telling the GIMP or Chrome or Apache our wishlists. In the long run these collaborations may also lead to getting more volunteers in WM spaces, which would be great. But in the near term, we ought to help keep bug reports and wishes from falling through the cracks.

[1]