Wikimedia Platform Engineering/MediaWiki Core Team/Quarterly review, October 2014
 July 2014 |
Wikimedia MediaWiki Core Team review October 2014 | January 2015 |
Previous quarter (Jul-Sep)
[edit]From the annual Wikimedia Engineering goals doc
We didn't quite make the goal of being substantially complete by the end of the quarter, but we are not far off from this point. We still anticipate there will be a long tail of niche services that rely on obscure Zend-engine features that may take some time to finish off, but we don't anticipate a need to maintain the current level of effort and focus on this project once the migration is substantially complete.
We continue to implement a new stack of technology:
- Migrate from PHP 5.3 to HHVM
- Upgrade from Apache 2.2 to Apache 2.4 (part of the Trusty upgrade)
- Upgrade from Ubuntu Precise (12.04) to Ubuntu Trusty (14.04)
The HHVM team at Facebook has been extremely helpful in getting to this point.
We intend to finish the engineering work in the upcoming quarter, and announce a date for the finalisation (which may occur sometime after the end of the quarter).
We indeed ran into the scaling issues we feared we would during the quarter, but all but the largest wikis are running CirrusSearch. Hardware is being purchased to finish the job.
Previous Quarter People Allocations
[edit]Our people allocations for July through September of 2014:
- Aaron Schulz: HHVM, general performance
- Ori Livneh: HHVM, performance metrics
- Tim Starling: HHVM, Architecture/RFC Review, other review
- Chad Horohoe: Search
- Nik Everett: Search
- Bryan Davis: SUL finalisation (time permitting: Deployment Tooling, LogStash, HHVM)
- Chris Steipp: SUL finalisation, Security reviews
- Dan Garry: SUL finalisation, SecurePoll, Search
- Brad Jorsch: Data & Developer Hub, API Maintenance, Scribunto Maintenance
- Antoine Musso: see Wikimedia Release and QA Team
- Greg Grossmeier: see Wikimedia Release and QA Team
- Sam Reed: see Wikimedia Release and QA Team
Upcoming quarter (Oct-Dec) primary projects
[edit]Library infrastructure for MediaWiki
[edit]The goal of this project is to show a path to decomposition of MediaWiki into smaller pieces. We will do this by peeling a pervasive set of functions out of core, organize those functions into an independent module (or replace the functionality with an externally developed module), and build the infrastructure to incorporate this module and other modules back into MediaWiki cleanly. Our long term goal is to make MediaWiki an application that is composed of many small purpose-built libraries (internally and/or externally developed) with interfaces that allow individual libraries to be exchanged for others.
Why do this?
- Make life better for new (and experienced) developers by organizing the code into simple components that can be easily understood.
- Reverses inertia toward ever expanding monolithic core by encouraging developers (in core) to develop their work as reusable modules with clearly-defined interfaces
- Starts making true unit testing of core viable by having individually testable units
- Provides an interim step on the way to service-oriented architecture in a way that is useful independently of that goal
- Encourages reuse and integration with larger software ecosystem. Done correctly, this will provide a useful means of expanding our development capacity through recruitment of library authors eager to showcase their work on a top 10 website.
- Share our awesome libraries with others and encourages contributions from them even if they aren't particularly interested in making our sites better.
Why now?
The longer we wait, the more workarounds and cruft we accumulate.
What will we do?
- Establish the precedent for how libraries should be incorporated into MediaWiki by replacing existing functionality with an existing library, such as logging (see Structured Logging RFC)
- Introduce some dependency management system to MediaWiki which can be used to configure complex objects
- Complete infrastructure for using composer to manage internal and external dependencies (WMF Composer RFC)
- Stretch goal: Establish the precedent for how libraries should be used by demonstrating the split of at least one library out of core (e.g. ResourceLoader). This may require participation from outside the MediaWiki Core team.
Editing performance
[edit]Concrete deliverable:
- Instrumentation of VisualEditor
- Much of this is done. Timing data is being emitted from ...
- Needs
- Rigorous definition of what needs to be tracked
- Correlation of changes in code to changes in performance
- Milestone:
- Weekly report sent out automatically from the system that is interesting, clear and actionable
- How does it align with the goals of the organization?
- 2014-2015 Engineering goal: "Understand, measure, and monitor readership across projects."
- How does it align with other team's commitments?
- TechOps wants to make metrics a priority for the coming quarter, so there's opportunity for collaboration there.
Secondary projects for MediaWiki Core in Oct-Dec
[edit]- Search result optimization - there's a lot of work that can be done to improve our search results, especially in the case of misspellings and thinkos. Once we get ElasticSearch deployed, an important next step will be to make it better.
- SecurePoll redesign completion - logging improvements, key handling improvements, and parameter editing still need to be implemented.
- API architecture - Brad has done more work on the API roadmap RFC, and has also made it possible to render our API documentation as clean HTML with proper class definitions for styling. He plans to continue this work after finishing up with the SecurePoll redesign.
Projects considered, but postponed or reevaluated
[edit]A/B testing
[edit]The team recognizes the importance of this work, and is very receptive to making this a priority in future quarters this fiscal. We are interested in helping to define this work, and have already put quite a bit of thinking into this.
Dependency injection/Setup.php
[edit]This project would involve establishing a precedent for using dependency injection in Setup.php, and having lazy initialization generally. This would clean up many messes; for example, user initialization is a mess because of the way things currently work. We decided we wouldn't pursue this as an end unto itself, but instead do the work in this area necessary to break out functionality into libraries.