Wikimedia Engineering/2015-16 Q3 Goals

Q2

Wikimedia Engineering Goals, FY2015-16 Q3 (January–March 2016)

Q4

Purpose of this document: Goals for the Wikimedia Engineering and Product Development department, for the third quarter of fiscal year 2015–16 (1 January 2016 – 31 March 2016). The goal setting process owner in each section is the person responsible for coordinating completion of the section, in partnership with the team and relevant stakeholders.

Status fields can use the following templates: In progress, To do, N Postponed,  Done or Partially done

ETA fields may use the initialism EOQ (End of Quarter).

Product

Community Tech

Objective	Key result	Dependency	ETA	Status
Partner with community on identifying and prioritizing needed improvements	Report back to community on analysis of top 10 wishes from Community Wishlist Survey on wiki, Phabricator and mailing lists: Status report #1 Host on-wiki discussions on refining the definitions of reported problems, validating proposed solutions		EOQ	Done
Improve core contributors' productivity	Complete development and ship features and fixes related to two wishes in the Wishlist Survey top 10: Migrate dead external links to archives Pageview stats tool		EOQ	Done

Discovery

We build the anonymous path of discovery to a trusted and relevant source of knowledge.

Our focus will be on building on-top of our anonymous and trusted search stack to improve relevancy, expose additional content and leverage locally relevant knowledge.

Goal setting process owner: Dan Garry

Search

Objective	Key result	Dependency	ETA	Status
Improve relevance of intra-wiki search results.	Roll out ElasticSearch completion suggester as a beta feature on all wikis early in the quarter. Depending on user feedback, roll out ElasticSearch completion suggester to production on all wikis late in the quarter. Review current ElasticSearch configuration, and use relevance lab to run tests to optimise the configuration to improve search result relevance. Integrate page view data into search result rankings to improve result relevance.	Operations: Wikimedia Labs (maintaining Labs) Discovery Analysis Team	March 2016	Done
Generate a model for user satisfaction with search results based on qualitatively-validated quantitative data.	Run a survey on-wiki which asks users who came from search if they found what they were looking for. Tie the responses from this survey back to clickthrough from search queries to validate the user satisfaction metric.	Operations: Wikimedia Labs (maintaining Labs) Discovery Analysis Team Reading Department (Extension:QuickSurveys)	March 2016	N Not done
Make www.wikipedia.org a portal for exploring open content on Wikimedia sites.	Run at least 3 A/B tests aiming to decrease the page's 60% bounce rate, by increasing the 30% search rate and 10% language link click rate. Graduate at least 1 successful A/B test to production on the portal.	Operations: Wikimedia Labs (maintaining Labs) Discovery Analysis Team	March 2016	N Not done

Wikidata Query Service

Objective	Key result	Dependency	ETA	Status
Support ongoing stability and add ability to support geocoordinate functionality by upgrading Wikidata Query Service to Blazegraph 2.0.	Upgrade to Blazegraph 2.0. Figure out what work we need to do to support geocoordinate functionality in Wikidata Query Service.	Release of Blazegraph 2.0 Operations	March 2016	Done

Maps & Geo

Objective	Key result	Dependency	ETA	Status
Improve the content discovery experience by switching the default tile service to the Wikimedia Maps Tile Server on all Wikivoyages.	Encourage most active Wikivoyage contributors to switch to Wikimedia maps tile server. Change default tile service for Wikivoyage maps across all languages to the Wikimedia maps tile server. Increase unique users of Wikimedia maps tile server.	Operations: Wikimedia Labs (maintaining Labs) Discovery Analysis Team	March 2016	Done

Analysis

The Discovery Department's Analysis Team leads the Discovery Department in experimental design, data collection, and data analysis. This quarter the team will not have its own goals, instead allocating itself as required to achieve the goals for Discovery listed in the other subsections.

Editing

Goal setting process owner: User:Trevor Parscal (WMF)

The Editing Department builds collaborative, inclusive tools for creating and editing free knowledge.

Objective			Key Result	Dependency	ETA	Status
Team	Objective	Outputs
Collaboration	Increase power and utility of the notification system	Roll out cross-wiki notifications	Users can access their notifications from all content wikis on every content wiki	—	EOQ	Done
Language	Improve reliability	Users should be confident that the tool is not going to fail unexpectedly resulting in waste of time or considerable extra work by fixing high-priority bugs that prevent the basic functionality like opening, saving and publishing translations, or heavily increases the maintenance effort by the community.	Acceptance from a community to move out of beta	QA support, Parsing, Tech Ops	EOQ	Partially done
Multimedia	Inform future decision making and improve upload tools	Gather data about users' success using various methods to upload, edit and use media Fix some of the issues in the upload tools that we detect through this work	The team has information about image use on pages, pages illustrated, first-time uploaders, total uploaders, etc. Editors have increased success levels in uploading through each tool	Editing Product	EOQ	Done
Parsing	Improve mobile reading experience and draw closer to Parsoid HTML for read views	Store the data-mw attribute in Parsoid HTML in a separate bucket in RESTBase Fetch this attribute separately in Visual Editor, Content Translation, OCG	Parsoid HTML sizes that are far more closer to PHP parser HTML sizes Reading web happier with leaner output	Services, Visual Editor, Content Translation, OCG	EOQ	Partially done
Product	Measure product acceptance	Provide support for an A/B test of providing access to VisualEditor by default to some anonymous users	Know more about how anonymous users react in practice to having access to VisualEditor by default	Research & Data	EOQ	Partially done
VisualEditor	Increase use of the visual editor	Release the single edit tab integration for the visual editor for all users on several Wikipedias Provide the visual editor as the non-default editor for anonymous users on the English Wikipedia Run an A/B test of providing the visual editor as the default editor for anonymous users on the English Wikipedia	More editors use the visual editor	Community Liaisons	EOQ	Partially done

Fundraising Tech

Goal setting process owner: User:AGomez_(WMF), User:Khorn_(WMF)

Objective	Key result	Dependency	ETA	Status
Increase level of PCI compliance for fundraising cluster	PCI SAQ A-EP complete	Ops support Security support 3rd party consultants appropriately scheduling and completing work	March 2016	Partially done
Support fundraising in Spanish speaking LATAM countries	Payment processing via Astropay ready to go for April campaigns by fr-online	fr-online fr-ops Astropay integration support	March 2016	Done
Start PayPal re-integration	Start PayPal upgrade (no intermediate goal yet defined)	PayPal integration support	March 2016	Done
Donor database cleanup: First Phase	Civi manual dedupe merges are reversible	donor services major gifts	March 2016	Done
Reduce SPOF in donation pipeline	Make all necessary code changes to prepare to replace ActiveMQ with Redis Set target date to do the replacement in production 2 engineers were working on the Education program for the month of February. In order to accommodate this, we have postponed working on this goal.	Jeff Green fr-online	March 2016	N Postponed

Reading

See planning doc for more context: Reading/Quarterly Planning/Q3

Objective	Key result	Dependency	ETA	Status
Web: Make Wikipedia more accessible to 2G connections with fast API-driven web experience in mobile web beta T113066)	Reduce data usage in mobile web beta by 20% Reduce first paint / time to interact to 5 seconds or less at the median article size and to 10 seconds or less at the 90th percentile article size on simulated / controlled 2G connections (lag excluded)	Critical path: Security, Architecture, Performance, Services, Parsing Less time involved: TechOps, Fundraising Tech, Release Engineering, Partnerships, Analytics	EOQ	Partially done
Web: Language switching improvement - to mobile web beta on all wikis (T121919)	Increase interaction with language switching by 5% on mobile web		EOQ	Done
iOS: Update Data Layer (T94037)	Improved load times, reduced crashes, increased feature dev velocity.		EOQ	N Postponed
iOS completed goal from Q2 Experiment: Drive app retention via feeds and modern UX (notifications Q3)	iOS App overhaul released 7 day 15% of users retained (up from 10% in Sept ‘15) (Feeds T104415, ModernizationT104510)	Design Research	EOQ	Done
Android: Sync Saved Pages in the app within a Wikipedia account, so they can be synchronized across devices (T120107)	Lay down the groundwork for the general implementation of "collections" in the Android app	Gather API (possible guidance requirements from Mobile Web team)	2016-02-15	N Postponed
Android: User can create a collection of articles (T120108)	A basic implementation of collections (visible and manageable only by the user), to set the stage for creating fully "social" collections.	Gather API (possible guidance requirements from Mobile Web team)	2016-04-01	Done

Additional iOS app notes

Continue to work on improving the retention rates for the app, and encourage repeat app usage with new experimental features. Particularly by implementing some user notifications. Initial notes on notification concepts here.

Additional Web notes

Aside from above goals, will support Android's efforts on Gather and hope to push 'read more' from beta into stable on as many wikis as possible.

Strategic tests

Reading strategic test work and research, such as a redirection test (relates to contemporary UX), is planned to be conducted in parallel with the items in tables above.

UI Standardization team

Objective	Key Result	Dependency	ETA	Status
Basic stylesheet of accorded design properties	Define workflow for requests on extending it by web-facing teams and community & pending documentation Initiate implementation of basic stylesheet in at least 3 WMF products Extending basic stylesheet properties by 25% Research into integrating continuous accessibility testing on properties and accompanying HTML	Possible implementation partners like Discovery, Editing or Reading Web Community	EOQ	Partially done
Style Guide – on how/when to use components	Increase consistency of UI components within living style guide by 25% with base.css implementation A holistic understanding of what our users' workflows, needs and wants in the style guide with 2 more focus groups done. Synthesized user research findings on designers and developer needs from WMF and community Presentation of page designers focus groups' results	Community members WMF designers Front-end developers involved in UI User Research team.	EOQ	Done Carried into Q4

Technology

Group goal

Objective	Key result	Dependency	ETA	Status
Make codfw (Dallas data center) functional as a backup data center	Serve MediaWiki application server traffic from codfw for 48 hours Serve Swift, ElasticSearch and RESTBase, Parsoid services from codfw	Technical Operations Performance (partial MediaWiki multi-DC work, Swift, MediaWiki configuration) Release Engineering (Scap, MediaWiki configuration) Services (RESTbase, Parsoid) Discovery (Search)	EOQ	Done Carried into Q4

Architecture

Goal-setting process owner:  Rob Lanphier

Objective	Key result	Dependencies	ETA	Status
Focused support for Wikimedia Security, assisting with hiring and implementing experiments such as assembling a security-focused working group (T123606) and creating a workable framework for security postmortems (T123753)	Scaling our security expertise	Emergent leadership and wide participation	EOQ	Done Much of the work was done by Darian Patrick, who stepped up after Chris Steipp's departure. I helped Wikimedia Security with bug triage meetings where I could.
Implement Fellowships program	Clear guidance from our most senior engineers, while giving them the autonomy to be effective for the movement	Editing	EOQ	N Postponed - I intend to regroup and discuss this with our future Wikimedia Foundation CTO

Analytics

Goal setting process owner: Nuria Ruiz

Feedback can be sent to analytics@lists.wikimedia.org (public e-mail list)

Objective	Key result	Dependency	Outcome/Rationale/Value Proposition	Status
Use Hadoop infrastructure to replace computation of Wikistats browser reports (http://stats.wikimedia.org)	Replace Browser reports and Viz - task T69053			Done Deployed to [1]
Operational Excellence for Pageview API/Eventlogging/EventBus	Work on this regard depends on operational load, some examples: Keep lights up (alarms, dashboards and monitoring) Usage reports Software upgrades (CDH5) Software deprecations (Limn, old puppet configurations) Eventlogging DB Maintenance Support Services team with EventBus work			Done
Deliver unique devices per wiki project daily and monthly via last access cookie	Validate computation method [2] and publish data to HDFS task T88647	Research		Done
Sanitization of Pageview Data	Pageview sanitization algorithm strategy peer reviewed with research and security	Research		To do

Release Engineering

Goal setting process owner: Greg Grossmeier

All tracked in: #releng-201516-q3 - More details at Wikimedia Release Engineering Team/Goals/201516Q3

Key Performance Indicators

Objective	Key result	Dependency	ETA	Status
STRENGTHEN Consolidate deploy tools	Migrate MediaWiki to scap3 - task T114313	Operations	EOQ	N Not done
FOCUS Retire Gerrit in favor of Phabricator	Integrate Differential with our Continuous Integration infrastructure - task T31 Shepherd the RFC - task T119908 Discuss at WikiDev16 - task T114320 -  Done Stretch: Garner early adopter projects (goal: 1 project per WMF "team")	WMF Engineering MW Tech community	EOQ	N Not done
STRENGTHEN Reduce CI wait time Wait time KPI	Migrate remaining CI jobs to Nodepool - task T119138 php composer (Zend and HHVM) - task T119139 as many miscellaneous jobs as possible - task T119140 javascript npm jobs - task T119143	Operations (minimal)	EOQ	N Not done

Services

Goal setting process owner: Gabriel Wicke

On phabricator: https://phabricator.wikimedia.org/T118868

Objective	Key result	Dependency	Status
STRENGTHEN API & community build-out.	Refinined guidelines and documentation to guide new service developers with less one-off mentoring. Support apps and the API-driven frontend effort with cacheable high-traffic entry points and the definition of content composition interfaces.	Reading Editing, especially the Parsing team Minor: Operations	Done
FOCUS Storage scaling & selective replication for hot data.	Reduce cost of history storage by improving compression ratios for HTML content to < 10% of raw content size. Support selective replication of hot data to edge PoPs for low-latency API access.	Minor: Operations	Done
EXPERIMENT Reliable event production & change propagation.	Implement reliable event production from MediaWiki to EventBus. Migrate RESTBase update jobs & cache updates to EventBus consumers. Provide a light-weight EventBus implementation for third party users. Prototype fine-grained dependency tracking for change propagation.	Minor: Operations	N Postponed

Technical Operations

Goal setting process owner: User:Mark Bergsma (WMF)

Objective	Key result	Dependency	ETA	Status
FOCUS Make codfw (Dallas data center) functional as a backup data center	Serve MediaWiki application server traffic from codfw for 48 hours Serve Swift, ElasticSearch and RESTBase, Parsoid services from codfw Ability to deploy MediaWiki, RESTbase, Parsoid from codfw	Performance (partial MediaWiki multi-DC work, Swift, MediaWiki configuration) Release Engineering (Scap, MediaWiki configuration) Services (RESTbase) Discovery (Search) Parsing (Parsoid)	EOQ	Done
STRENGTHEN Move basic Labs administration workflow from wikitech to Horizon	2FA Horizon authentication Support instance manipulation (create/delete) Allow modification of Labs public DNS records Allow manipulation of HTTP Proxies	Security (authentication backend)	EOQ	Done
STRENGTHEN Eliminate Ubuntu 12.04 Precise systems in our infrastructure (technical debt)	Reduce amount of remaining Ubuntu 12.04 systems from 100 to <= 40		EOQ	Done
STRENGTHEN Improve access to and control over incident and metrics monitoring infrastructure	Evaluate alternative web interfaces  Done Migrate to a web interface with improved access control over service checks and their stateN Not done Implement email & paging alerts for all service "owners" outside TechOpsN Not done Consolidate Graphite metrics monitoring frontends into grafana Done		EOQ	Partially done
EXPERIMENT Stretch: Evaluate and test limited deployment of Varnish 4	Evaluate basics of Varnish 3.x -> 4.x migration for existing functionality Convert at least one cache cluster to Varnish 4.x		EOQ	Done

Performance

Goal-setting process owner: Ori Livneh

Speed is Wikipedia's killer feature. ("Wiki" means "quick" in Hawaiian.) As the Wikimedia Foundation’s Performance team, we want to create value for readers and editors by making it possible to retrieve and render content at the speed of thought, from anywhere in the world, on the broadest range of devices and connection profiles.

The current primary focus of the team is availability.

Like many web applications of its class, MediaWiki was not designed from the outset to be a distributed system. The features that make MediaWiki reliable and scalable were typically developed in response to pressure generated by the growth of Wikipedia. As a result, although Wikimedia has multiple data centers across the globe, MediaWiki is only running on one of them at any given moment. The result is that users around the world who have access to culture and knowledge that is currently underrepresented in Wikipedia are hit with very high latency whenever they try to contribute. We want to fix that.

The following is a summary of the Performance Team Goals.

Objective	Key result	Dependency	ETA	Status
STRENGTHEN T66214 Content-based addressing of images.	Migrate images to a URL scheme based on a hash of the file contents. Allow images to be cached by the browser.	Operations	EOQ	In progress
FOCUS T119798 Leaner mobile web.	Segment the mobile web site into a high-bandwidth and a low-bandwidth variant. Route readers to the best variant based on ther connection speed.	Reading.	EOQ	In progress
EXPERIMENT T117411 On-wiki performance inspector tool.	Share performance data with editors, so they know how the article they are on will load on different connection profiles. Provide per-wiki leaderboards showing the most expensive Scribunto modules and the pages which are slowest to parse. Make ourselves (the performance team) obsolete!	Community.	EOQ	In progress

Research

In Wikimedia Research we use qualitative and quantitative methods to provide strategic insights and technological solutions to the movement and the Foundation, to foster innovation and to inform the development of new products.

Research & Data

Goal setting process owner: Dario Taraborelli [ Wikimedia Research goals overview ]

Objective	Key result	Dependency	ETA	Status
STRENGTHEN Extend revision scoring to support new models and languages	Develop and test edit typing model (T121715 In progress) Tune damage detection model (T120138  Done) and extend current models to 5 other Wikipedia language editions (T121712 In progress) Write and submit papers (KDD, CSCW) (T121718  Done - T121719 In progress)	Ops (Labs)	EOQ	In progress
EXPERIMENT Run and analyze a content creation campaign	Run an article creation campaign in the wild, teaming up with volunteers to improve the design and performance of recommendations. (T121962)	None	EOQ	N Not done
FOCUS Reader segmentation research	Run a reader survey integrated with log analysis to identify segments in the Wikipedia reader population and their reading behavior. Analyze and present the results (T121727)	Reading	EOQ	Done

Design Research

Goal setting process owner: Abbey Ripstra [ Wikimedia Research goals overview ]

Objective	Key result	Dependency	ETA	Status
FOCUS Production Work	Provide generative and evaluative design research in support of human centered product development. (board) Contextual Inquiry in Mexico Evaluative research when needed by teams Generative interviews on going	Being able to collaborate with product teams. Budget from Reading team for contextual inquiry in Mexico.	EOQ	Done
STRENGTHEN Persona Iteration	Continue to iterate our set of pragmatic personas into personas informed by qualitative research. (board)	Participants showing up to interviews.	EOQ	In progress
STRENGTHEN Mentoring non design researchers	Mentoring non design researchers to do quality design research. (board) Sherah Smith in Reading UI Standardization team (May Tee and Volker Eckl) Program Toolkits Team	Willing and able mentees.	EOQ	Done
EXPERIMENT External Collaborations	External Collaboration with University of Washington: Better understand university students and their needs around learning online. (T120760)	Collaboration with Professor and students at UW.	EOQ	In progress

Security

Goal setting process owner: CSteipp (WMF)

We make life hard for the people who want to harm our sites or the people that use them.

Objective	Key result	Dependency	ETA	Status
STRENGTHEN Reduce the liklihood of compromised user accounts, which could lead to vandalism or disclosing private data, by strengthening the authentication options available to wiki accounts.	Allow CentralAuth accounts to enable and enforce two-factor authentication for their accounts.	Implementation will need to account for AuthManager work by Reading Infrastructure	EOQ	In progress