Jump to content

Wikimedia Platform Engineering/MediaWiki Core Team/Quarterly review, July 2014

From mediawiki.org
April 2014 Wikimedia MediaWiki Core Team review July 2014 October 2014
Slides from review

This is an outline for a review of the MediaWiki Core team which took place at WMF on July 16, 2014.

Team

[edit]

See Wikimedia MediaWiki Core Team page.

Changes since last review: (none)

Previous quarter

[edit]

HHVM will be our flagship project. HHVM increases performance across the board on MediaWiki, and may eventually allow for the removal of the caching layer, thereby opening the doors for making the site more interactive and performant. In parallel with HHVM, the Search team will be focussing on getting CirrusSearch deployed on all Wikimedia wikis, and work will commence on the SUL finalisation.

The above work will continue alongside our team's ongoing responsibilities.

HHVM

[edit]

Great strides were made on HHVM. Almost all required extensions have been ported, and the remainder (e.g. libxml) are very close to finished. Packaging for Debian/Ubuntu is done thanks to a big effort in Ops (led by Faidon Liambotis), which makes wider deployment much practical. Occasional production testing is happening on a single host ("osmium") which has HHVM installed on it.

[edit]

CirrusSearch Deployment

[edit]

Deployed CirrusSearch as the primary search backend on all but 11 wikis (as of July 1). Spent time implementing a powerful power user feature that was a long standing request as well as more performance work.

Search page redesign

[edit]

This work was deprioritized this quarter.

SUL finalisation

[edit]

Kunal Mehta (Legoktm) didn't have as much time as we'd hoped for to work on this, but he was able to make some progress. Much of the groundwork was laid for a more concerted effort in the following quarter.

Scap

[edit]

Work continued on moving Scap fully to Python, which was largely completed. The need for HHVM bytecode distribution in scap was investigated, and determined to be unnecessary.

SecurePoll Redesign

[edit]

This work is wrapping up now, and should be ready for deployment in the next election.

Architecture/RFC Review

[edit]

RFC reviews are now a more routine part of the process. Though progress has been made on the backlog of RFCs, the number of incomplete RFCs is still rather large.

Previous Quarter People Allocations

[edit]

Our people allocations for April through June of 2014:

Upcoming quarter

[edit]

From the annual Wikimedia Engineering goals doc

HHVM

[edit]

We believe that HHVM deployment can be substantially complete by the end of the quarter (where "substantially" means we have migrated just about all of our application servers and many of our other PHP-based services). We anticipate there will be a long tail of niche services that rely on obscure Zend-engine features that may take some time to finish off, but we don't anticipate a need to maintain the current level of effort and focus on this project once the migration is substantially complete.

Our plan is to implement a new stack of technology:

  • Migrate from PHP 5.3 to HHVM
  • Upgrade from Apache 2.2 to Apache 2.4 (part of the Trusty upgrade)
  • Upgrade from Ubuntu Precise (12.04) to Ubuntu Trusty (14.04)

The HHVM team at Facebook has been very helpful so far, and is dedicating engineers to help us finish this work. We expect to work very closely with them in the coming quarter.

Quant target: By September 2014, we aim to halve median backend response time for uncached operations, including page preview and save.

Performance

[edit]

We had planned to establish key performance indicators for end-to-end response time and server performance. However, HHVM will absorb the full attention of the people most qualified to do this. Postponing until Q2.

Single user login finalisation

[edit]

We intend to finish the engineering work in the upcoming quarter, and announce a date for the finalisation (which may occur sometime after the end of the quarter).

Search

[edit]

The team is now reasonably confident that, with a minimal investment in hardware (or even clever shuffling of hardware), the migration can be completed by the end of this quarter, though English Wikipedia could bleed over into October if we run into scaling issues there. They have published a schedule, halting progress if we discover enough high impact bugs to keep Nik and Chad busy with fixes prior to the next scheduled deployment.

Architecture/RFC Review

[edit]

Data & Developer Hub

[edit]

Developers who interact with our sites need better guidance to use our APIs

Goals:

  • Developer Hub prototype
  • Landing page, 3 projects showcased, 3 APIs documented
  • API sandbox functional prototype
  • Clear process for future improvements

Upcoming Quarter People Allocations

[edit]

Our people allocations for July through September of 2014:

Ongoing responsibilities

[edit]
  • Deployments
    • Core deployments
    • External team deployments (e.g. Wikidata)
  • MediaWiki operations (performance, debugging, ops team support)
  • Code review
    • API maintenance and code review (Brad: 30%)
    • Security (CSteipp: 20%)
  • Security issue response
  • Test infrastructure (Beta cluster and continuous integration)
  • Git/Gerrit improvement
  • Shell bugs
Retrieved from "https://www.mediawiki.org/w/index.php?title=Wikimedia_Platform_Engineering/MediaWiki_Core_Team/Quarterly_review,_July_2014&oldid=2802787"