Scrum of scrums/2017-04-05

contact: https://www.mediawiki.org/wiki/Wikimedia_Engineering

Call outs

• Analytics: Planning wikistats 2.0 prototype consultation - Please share your thoughts on the UI prototype https://analytics-prototype.wmflabs.org (emails with where to share soon)
• Parsoid drops node 0.10 support -- announcement and details coming this week.
• ParserMigration extension now live on all wikis -- announcement and details coming next week.
• Services: New version of the PDF render service still awaits checking from WMDE
  • https://en.wikipedia.beta.wmflabs.org/api/rest_v1/#!/Page_content/get_page_pdf_title
  • https://phabricator.wikimedia.org/T160764
• RelEng: May need Ops help deploying 3d2png via scap3 (worked on Beta Cluster), need some +2 in puppet
• FR-tech's new job runner / cron complement script to replace our Jenkins instance is working like a charm. Anyone else want to use it for scheduled jobs?
  • https://github.com/wikimedia/wikimedia-fundraising-process-control

Product

Reading

Web

• Last week:
  • Reading Off-site
• Next week:
  • Deploy PagePreviews to Hungarian and Hebrew wikipedias.
  • PagePreviews instrumentation related bugfixes.
  • Improved print styles for MobileFrontend

Android

• Last week:
  • Reading Off-site.
  • A nasty little crash in production for logged-in users caused by a small, unexpected change to the userjs response format. ()
• This week / early next week: a new release to beta and production thereafter, including the following:
  • A substantially better offline experience! (our Q3 goal: https://phabricator.wikimedia.org/tag/android-app-goals-2017-jan-mar-q3/)
  • Wikidata description editing expanded from three languages to all except the top 10 by page views as measured in the app.
• Kanban: https://phabricator.wikimedia.org/project/view/2352/

Reading Infrastructure

• TemplateStyles:
  • CSS sanitizer library https://gerrit.wikimedia.org/r/#/q/project:css-sanitizer
  • will be taken over by MediaWiki Platform
• Mobile Content Service: https://phabricator.wikimedia.org/project/view/2445/
  • Deployed: Added 13 more languages to the featured article.

iOS

• Last Week
  • Reading Offsite
• This week
  • 5.4.1 - https://phabricator.wikimedia.org/project/view/2600/
    • Monitor crash reports, fix major issues
  • 5.5 - https://phabricator.wikimedia.org/project/view/2602/
    • Places
    • JavaScript consolidation with Android
    • Image caching fixes

Community Tech

• Cookie blocks deployed to English Wikipedia, more wikis to follow
• LoginNotify in testing on beta cluster
• CodeMirror bug fixes, colour tweaks
• Various improvements to the new XTools, nothing to show
• No blockers

Editing

Parsing

• ParserMigration extension has now been deployed on all wikis. You can enable it on the Editing tab of your Special:Preferences page. Clicking it will add a sidebar link called "Edit with migration tool". This opens a page for editing that shows you a side-by-side preview of the page with the default (production) parser and an experimental / proposed-migration of the parser. For example, currently, this lets you compare output with Tidy and output with RemexHTML (a PHP-only replacement for Tidy that is based on HTML5 semantics). You should also be able to directly exercise it via https://en.wikipedia.org/w/index.php?title=PAGE-TITLE&action=parsermigration-edit. For example, https://en.wikipedia.org/w/index.php?title=Typhoon_Parma&action=parsermigration-edit shows that the page renders differently with RemexHTML ( because of https://phabricator.wikimedia.org/T161341#3129283 ). We will send out a note on mailing lists (and possibly other forums like Tech News) next week about this and about how to leverage the Linter extension and Visual Diffing results for fixing wiki pages
• Filed https://phabricator.wikimedia.org/T162275 to make visual diffing output more useful for editors for fixing up pages. This can help us track progress of wikitext and template fixup and tell us when it is safe to flip the switch on Tidy.
• We are continuing to work on https://www.mediawiki.org/wiki/Parsoid/Language_conversion/Preprocessor_fixups to document impact of cleanup to the preprocessor to remove some edge cases ( https://gerrit.wikimedia.org/r/#/c/333997/ ). We'll use the results of this investigation to figure out followup work.
• We released Parsoid debian 0.7.0 package yesterday. This release drops support for node 0.10x and requires node v4x. However, we forgot to update the dependencies in the control file which still states a >= v0.8.0 requirement. Will release a 0.7.1 deb that fixes this.
• In November, we announced a deprecation of node 0.10 for Parsoid ( https://lists.wikimedia.org/pipermail/wikitech-l/2016-November/087005.html ), Going forward, node 0.10 is no longer supported in Parsoid.
• Bumping service-runner in Parsoid as requested by Services team for the upcoming DC switchover this month.

Collaboration

• RCFilters beta feature went live on plwiki, ptwiki last week; next release to trwiki, fawiki, ruwiki and frwiki next week (Apr 11); then enwiki + most non-ORES wikis on Apr 24; and all remaining wikis May 9
• Working with Aaron H on figuring out ORES behavior for RCFilters for the next releases

Language

• Work on OOjs UI migration continue.

Technology

Analytics

• Reportcard migrated to new infra - Limn is dead
• new AQS metrics (historical projectcounts) have a bug - still vetting before announcing
• user agent map in EventLogging replaces user agent string (fully deployed)
• All hadoop workers have been reimaged to jessie - masters left (next week)
• Planning wikistats 2.0 prototype consultation (https://analytics-prototype.wmflabs.org)
• Eventlogging MariaDB custom replication procedure has been updated (should avoid long delays between master and slave)
• db1047 is to be decommissioned - https://phabricator.wikimedia.org/T156844#3107357

Security

• Reviews:
  • iOS 2-factor wrapping up
  • Ex:WikibaseMediaInfo
  • Ex:CodeMirror

Services

• Blockers: none
• Updates:
  • New version of the PDF render service still awaits checking from WMDE
    • https://en.wikipedia.beta.wmflabs.org/api/rest_v1/#!/Page_content/get_page_pdf_title
    • https://phabricator.wikimedia.org/T160764
  • All SCB services have been redeployed in preparation to DC switchover to pick up short-term DNS caching

RelEng

• Blockers:
  • May need Ops help deploying 3d2png via scap3 (worked on Beta Cluster), need some +2 in puppet

Discovery

• No blockers
• Discussing strategy for enabling live updates for completion suggester - probably starting with new pages
• New Polish analyzers deployed, Ukrainian is next, working on Chinese
• Working on Wikidata search improvement
• Working on adding non-exact search capabilities to Special:Undelete
• Working on improving noop detection plugins for Elasticsearch
• Preparing for active-active switch

Fundraising Tech

• Moved all our scheduled jobs off Jenkins with surprisingly little complication: https://phabricator.wikimedia.org/T161569
  • Anybody need a job scheduler / cron wrapper script?
  • yaml job description files with command, timeout, environment vars
  • Ensures jobs run as a specified user
  • non-ops can be given the ability to run defined jobs on demand
  • can generate cron.d entries automatically
  • See more here: https://github.com/wikimedia/wikimedia-fundraising-process-control
• More work on Central Notice banner sequence, and general admin UI refactor: https://phabricator.wikimedia.org/T144453