Scrum of scrums/2018-08-29

2018-08-29

Callouts

• SRE:

Switchover/Switchback dates reminder. It's in 2 Weeks.

Services: Tuesday, September 11th 2018 14:30 UTC Media storage/Swift: Tuesday, September 11th 2018 15:00 UTC Traffic: Tuesday, September 11th 2018 19:00 UTC MediaWiki: Wednesday, September 12th 2018: 14:00 UTC

Switchback:

Traffic: Wednesday, October 10th 2018 09:00 UTC MediaWiki: Wednesday, October 10th 2018: 14:00 UTC Services: Thursday, October 11th 2018 14:30 UTC Media storage/Swift: Thursday, October 11th 2018 15:00 UTC

IMPORTANT: For the duration of the switchover (1 month), please refrain from large db schema changes and please avoid deploying any kind of new feature that requires creation of tables. IMPORTANT: There will be a train freeze in the week of Sept 10th and Oct 8th.

• Fundraising campaigns https://meta.wikimedia.org/wiki/CentralNotice/Calendar
• From RI: help wanted with https://phabricator.wikimedia.org/T203029 from those who know about Dashiki

[nuria] Dashiki just uses the files that are now not rendering from meta, it is really a seprate system that consumes those files

• wp10 models in ORES API are renamed to articlequality, if you're using this model, please change
• Wikidata would appreciate input/decision on way forward with two patches in MW core, related to testing (thanks to Anotine and Timo for input so far)
  • https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/455176/
  • https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/446543/

Audiences

Contributors

Community Tech

• Blocked by:
• Blocking:
• Updates:

Anti-Harassment Tools

• Blocked by:
• Blocking:
• Updates:
  • Continue work on Partial Blocking (Goal is to have a MVP ready by the end of the sprint)
  • Analytics on Harassment cases on A/NI

Editing

• Blocked by:
• Blocking:
  • Updates:
  • Improving event logging for better data analysis
  • Mobile bug fixes to improve editing experience

Growth

• Blocked by:
• Blocking:
• Updates:

Language

• Blocked by: None
• Blocking: None
• Updates:
  • Work on ContentTranslation version 2 continues.

Readers

iOS native app

• Blocked by:
• Blocking:
• Updates:

Android native app

• Blocked by:
• Blocking:
• Updates:
  • Finished polling Echo notifications (limited to certain types of notifications initially)
  • Waiting on user testing of navigation prototypes.

Readers Web

• Blocked by:
• Blocking:
• Updates:

Readers Infrastructure

• Blocked by:
• Blocking:
• Updates:
  • Bernd out rest of week, no PCS updates
  • Maps:
    • Working on setting up map tile invalidation on resource change (via ChangeProp)
    • maps-test cluster to be decommissioned; production maps cluster to be updated to Stretch
  • JsonConfig bug affecting Dashiki config pages on meta.wikimedia.org, should be fixed but isn't. Help wanted! https://phabricator.wikimedia.org/T203029
    • https://gerrit.wikimedia.org/r/c/mediawiki/extensions/JsonConfig/+/455868 fixed the issue on zero.wikimedia.org (https://phabricator.wikimedia.org/T203006 ), but not for Dashiki config pages

Parsing

• Blocked by: None
• Blocking: None, coordinating with VE on several updates such as audio/video elements
• Updates: Offsite upcoming on 9/10-9/15

Continuing porting TokenTansformers subsystem to PHP as part of overall php port

Multimedia

• Blocked by:
• Blocking:
• Updates
  • SDoC: multi-lingual captions display and edit on File page
  • SDoC: prototyping depicts search almost complete
  • SDoC: design artwork

UI Standardization

• Blocked by:
• Blocking:
• Updates:

Technology

Analytics

• Blocked by:
• Blocking:
• Updates:
  • Updating privacy policy of wikistats2 and other sites like http://analytics.wikimedia.org to the new non-wiki policy per Legal’s request.
  • Working with performance on making eventlogging a lot more limber so it can be included in every page: https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/EventLogging/+/453457/
  • Changed all random identifiers in mediawiki codebase to have more entropy, from 64 to 80 bits. This benefits eventlogging but also it is used widely on mediawiki client code.: https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/454460/
  • Integrating archiva (java libraries dependency management depot) with LDAP.
  • Moving all our analytics infrastructure to Debian Stretch. Along the way, we will be moving (superset|turnilo|yarn|hue).wikimedia.org to new VMs and upgrading them.

Cloud Services

• Blocked by:
• Blocking:
• Updates:

Fundraising Tech

• Blocked by:
• Blocking:
• Updates:
  • More testing and bug fixes on main CC processor's new API integration
  • CiviCRM UI enhancements
  • Donor data export/erase bug fixes
  • Testing new pipeline to get banner and landing page impressions into fundraising DBs
  • Testing stretch and PHP7 to unblock payments mediawiki LTS upgrade
  • Getting started on hiring another fr-tech

MediaWiki Core Platform

• Blocked by:
• Blocking:
• Updates:
  • Platform Evolution: summarizing stakeholder input to cluster themes for Tech Conf
  • Requirements gathering and discussion for storage component
  • More work on deploying the phan security check (aka phan-taint-check) to more extensions
  • Porting poolcounterd tests to Python (gerrit:455385)
  • LinksUpdate failure on replag (phab:T201482)
  • EU copyright landing page

Performance

• Blocked by:
• Blocking:
• Updates:
  • mcrouter broadcast now live (deletes go to both data centers)
  • Thought we had made the world much faster by removing a ResourceLoader round trip, but then realized that we'd accidentally gamed our numbers. Still evaluating the actual improvement.
  • Gilles is back from vacation, and cranking through Thumbor bugs
  • Testing "hero images" timings.

Release Engineering

• Blocked by:
  • Noise from https://phabricator.wikimedia.org/T201082 during Train deployment (not really blocked but distracted)
  • Block the CI migration to Quibble / off of Nodepool
    • MediaWiki team: "[TrustedXFF] on fresh install trusted-xff.cdb is missing and User::getDefaultOptions() cause a fatal" https://phabricator.wikimedia.org/T198120
    • MediaWiki team: "[ReadingLists] No registered builder available for ReadingLists." https://phabricator.wikimedia.org/T196567
    • WMDE or a Qunit/JavaScript guru: ArticlePlaceholder should use MediaWiki qunit runner https://phabricator.wikimedia.org/T180171

• Blocking:
• Updates:
  • Two post mortems this week about recent train problems
  • Train: no major problems, 1.32.0-wmf.19 at group 0 https://phabricator.wikimedia.org/T191065 https://tools.wmflabs.org/versions/
  • Log spam: Unknown modifier 'R': [/^page\-User\:BeneBot.+/RfD\-open/text$/] in /srv/mediawiki/php-1.32.0-wmf.16/extensions/Translate/stringmangler/StringMatcher.php https://phabricator.wikimedia.org/T202058

Research

• Blocked by: None
• Blocking: None
• Updates:
  • Sent out emails to collect more data (2nd stage of the experiment) for the Elicit new editor interests research: https://meta.wikimedia.org/wiki/Research:Voice_and_exit_in_a_voluntary_work_environment/Elicit_new_editor_interests#Stats_and_study_condition
  • Will enable collecting data for the CitationUsage schema this Thursday: https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/454854/
  • Working towards productionizing the recommendation API. Talking about storage with Services/Analytics: https://phabricator.wikimedia.org/T203039

Scoring Platform

• Blocked by: RelEng on these two: https://phabricator.wikimedia.org/T192042 and https://gerrit.wikimedia.org/r/c/All-Projects/+/455629
• Blocking: None
• Updates:
  • The work for using PoolCounter in ores continues, support for multiple nodes is almost there
  • Renaming wp10 models to articlequality, if you're using this model, please change
  • Reducing size of ores setup in labs, to reduce the cloud service team maintenance workload
  • working on JADE table schemes

Search Platform

• Blocked by:
• Blocking:
• Updates:
  • Enabled indexing of all wikidata statements referencing items: https://phabricator.wikimedia.org/T199884 newly edits have it enabled, old items pending reindex
  • Added collecting stats for data processing daemons: https://phabricator.wikimedia.org/T201948
  • Fixed errors with large offsets on search: https://phabricator.wikimedia.org/T201360
  • Working on running multiple Elastic instances on the same hardware: https://phabricator.wikimedia.org/T193654
  • Working on searching for statement values without additional keywords: https://phabricator.wikimedia.org/T163642
  • Working on ES 6.3 upgrade: https://phabricator.wikimedia.org/T197960
  • Working on query parsing refactoring: https://phabricator.wikimedia.org/T185108

Security

• Blocked by:
• Blocking:
• Updates:
  • Starting work into using Content-Security-Policy header
  • Security review of chromimum-render https://phabricator.wikimedia.org/T177765 and EU copyright once ready

Services

• Blocked by:
• Blocking:
• Updates:

Site Reliability Engineering

• Blocked by:
  • None
• Blocking:
  • None
• Updates:

Switchover/Switchback dates reminder

Services: Tuesday, September 11th 2018 14:30 UTC Media storage/Swift: Tuesday, September 11th 2018 15:00 UTC Traffic: Tuesday, September 11th 2018 19:00 UTC MediaWiki: Wednesday, September 12th 2018: 14:00 UTC

Switchback:

Traffic: Wednesday, October 10th 2018 09:00 UTC MediaWiki: Wednesday, October 10th 2018: 14:00 UTC Services: Thursday, October 11th 2018 14:30 UTC Media storage/Swift: Thursday, October 11th 2018 15:00 UTC

IMPORTANT: For the duration of the switchover (1 month), please refrain from large db schema changes and please avoid deploying any kind of new feature that requires creation of tables. IMPORTANT: There will be a train freeze in the week of Sept 10th and Oct 8th.

Wikidata

• Blocked by:
  • Blocked is a big word, but we'd appreciate input/decision on way forward with two patches in MW core, related to testing (thanks to Anotine and Timo for input so far)
    • https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/455176/
    • https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/446543/
• Blocking:
• Updates:
  • Work in Lexicographical data continues:
    • adding senses
    • improving the editor workflow
    • https://phabricator.wikimedia.org/project/view/2292/
  • Experiment about not using secondary SQL table for formatting item links being gradually deployed
    • https://phabricator.wikimedia.org/T201831

German Technical Wishlist

• Blocked by:
• Blocking:
• Updates:

Multi-Content Revisions

• Blocked by:
• Blocking:
• Updates:
  • almost complete phase 2 tasks (today?), planning to finish phase 3 tasks by end of week
  • deployment impacted by CI test failures - remediation in progress

SoS Meeting Bookkeeping

• Updates: