Wikimedia Platform Engineering/MediaWiki Core Team/Check-ins/20140331

who: Brad, Ori, Aaron, Rob, Greg, Chris, Chad, Nik, Bryan, Dan, Tim regrets: no regrets, no apologies

Bug escalation

• https://bugzilla.wikimedia.org/show_bug.cgi?id=46014 that dang old inconsistent revision content from google indexers…
  • Brad taking it on
• Memcached traffic increase: was the result of https://gerrit.wikimedia.org/r/#/c/117231/ , but calls into question the strategy of relying on memcache to repeatedly retrieve an ~80k (in the case of enwiki) blob

Quarterly Review

Contenders:

HHVM: continuing work on that

• • Reasons AGAINST: good packages not expected before May; more work than could probably be done in a single quarter.
  • Reasons FOR: not being able to get to a full production roll-out by the end of the quarter does not mean that we don’t easily have a quarter’s worth of work to usefully invest. There are a lot of loose ends from this quarter.

Revamp revision storage (w/Gabriel & co.)

• Pro: would be a catalyst for refactoring things in core
• The task: (a) specify an API for storage of revisions; anything implementing that API could be trivially used with MediaWiki. (b) refactor core so that rev storage is an implementation of that API.
• Have an interface that is bundled with core and that provides all the required functionality.
• Gabriel’s revision storage platform would be an alternate implementation of that interface.
• TODO: Ping Gabriel and find out how much intends to do on his own and how a sprint by the core team could assist

Job queue work

• • Bug 46770 “Rewrite jobs-loop.sh in a proper programming language”? [TS]
    • Bug 46770 would be nice. So would more monitoring. I tend to agree with Rob that it’s probably not worth a full quarter (nor do I agree that we should scrap the whole thing in favor of $someNewThing) [CH]
    • Antoine: Isn’t our time better invested in overhauling the whole job queue system?
      • But WHY? Like Rob said...there doesn’t seem to be a compelling case for replacing the whole thing...just making some improvements around the edges. [CH]
      • Not really, after Aaron already spent so much time overhauling it over the last 18 months [TS]
        • +10 [CH]
    • We could use better monitoring of the queue processing, nicer priority systems..

SUL finalization

• Legoktm should be able to do the engineering work, Dan can do the other loose ends

OpenID connect (goes hand-in-hand with Phabricator)

• Continuous integration fully tied to Gerrit stream-events / Gerrit comment. Need to either adapt Zuul or rethink the way we do CI.
  • https://secure.phabricator.com/book/phabricator/article/herald/ Herald -> gerrit stream events adapter should be an easy hack.
    • Or we can just get rid of Zuul… (We should, but the ability to kludge things temporarily during a migration would be good.)

Scap

• Get to “100%” moved to Python

Performance

• Speed up LocalFile locking behavior (https://gerrit.wikimedia.org/r/#/c/122550/) (Aaron’s patch; Ori to review.)
• BagOStuff::lock in general should distinguish “locked” from “can’t connect”
• PDF thumbnail pool counter use (coming soon to TIFFs)
• GeoIP <script> → cookie fully deployed
• Ori, Timo & Roan working on frontend performance workshop for Zurich
  • Preliminary notes: https://etherpad.wikimedia.org/p/perfworkshop

HHVM

• http://en.hhvm.beta.wmflabs.org/wiki/Main_page
  • Missing static assets: https://bugzilla.wikimedia.org/show_bug.cgi?id=63310
    • Probably the result of loading a different set of extensions.
  • Required hacking a package in an ugly way; waiting for a fix in RT 7133
• Lightning talk for metrics meeting on HHVM with Flow as case-study (Ori + Erik B.)
  • Preliminary notes: http://etherpad.wikimedia.org/p/hhvm-lightning
• Patches needing review:
  • https://gerrit.wikimedia.org/r/#/c/117362/ (HHVM support for Wikidiff2)
  • https://gerrit.wikimedia.org/r/#/c/120180/ (Add a handler for HHVM fatals)
  • https://gerrit.wikimedia.org/r/#/c/120469/ (HHVM support for FastStringSearch)
• HHVM 3.0 released. http://hhvm.com/blog/4349/hhvm-3-0-0
  • 3.0 release brings exciting new headaches.
• HHVM discussed during today’s TechOps meeting; waiting to hear how it went.

Performance profiling for JavaScript code

• Something Ori will work on, but could be a bona fide project if Timo joins platform

Deployment tooling / RelEng

• Beta migration to eqiad complete!
• Upgraded elasticsearch and kibana for logstash cluster in prod and beta
• Fixed bug 62862 in scap (errors in python scripts were logged but the exit status was not properly returned to the shell)
• Still need those two config changes done before tomorrow(?)

Search

• Going to all group1 wikis but commons/meta/incubator as primary on Wednesday
• Elasticsearch 1.1.0 has been out for a week without another revision planned so I’m verifying against it
• Nicer highlighter on the horizon.
• This Thursday we’ll get some performance fixes on enwiki which we’ll use to run another round of performance tests
• Search page redesign slowing down this sprint while we wait for some designs from Brandon

SecurePoll

• Going to be moving the date picker into SecurePoll instead of core, apparently
• Other than that, Brad has been being pulled to other bugs recently

ContactForm for trademark

• Brad set up a Labs instance
  • We have the mediawiki-core-team project now! If Brad missed giving anyone access, everyone else is an admin and can add you.
• Dan has pinged Yana for feedback on the form.