Scrum of scrums/2017-12-06
Appearance
2017-12-06
[edit]Callouts
[edit]- Operations blocked on https://phabricator.wikimedia.org/T172025 (Flow)
- Operations DBAs s8 master switchover programmed for 9th January
- Reminder! This is your last few weeks of deployments for the year/quarter! No non-emergency deploys starts the week of December 18th.
- MediaWiki 1.30 is about to be released in the next few days, so last call for any blockers, phabricator tag is #mw-1.30-release
- Wikidata welcomes secondary review from somebody with knowledge about the recent "section editing" change: https://phabricator.wikimedia.org/T181807
- Have a look at newly initiated https://www.mediawiki.org/wiki/Manual:Coding_conventions/SVG. Feedback welcome!
Audiences
[edit]Readers
[edit]iOS native app
[edit]- Blocked by:
- Blocking:
- Updates:
- Releasing 5.7.2 (high priority fix for https://phabricator.wikimedia.org/T69015 ) ( https://phabricator.wikimedia.org/tag/ios-app-v5.7.2/ )
- Continuing work on 5.7.3 (Faster article loading, other minor enhancements) for release before the end of the year ( https://phabricator.wikimedia.org/project/view/2913/ )
- Continuing work on 5.8 (Reading Lists) for release next year ( https://phabricator.wikimedia.org/project/view/3131/ )
Android native app
[edit]- Blocked by:
- Blocking:
- Updates: Releasing beta 2.6.207
Reading Web
[edit]- Blocked by:
- Blocking:
- Updates:
Reading Infrastructure
[edit]- Blocked by:
- Blocking:
- Updates:
- ReadingLists performance improvements
- Wrapping up media + summary endpoints
- References needs a bit more design work
Multimedia
[edit]- Blocked by: N/A
- Blocking: N/A
- Updates: 3D nearly ready to go, probably before holidays, else shortly thereafter. Also working on prototyping for first feature of Structured Data on Commons.
Discovery
[edit]- Blocked by:
- Blocking:
- Updates:
Maps
[edit]- Blocked by: N/A
- Blocking: N/A
- Updates:
- Maps mediawiki integration (Kartographer) improvements #1 on community wishlist: https://meta.wikimedia.org/wiki/2017_Community_Wishlist_Survey/Tracking
- Redeploying Kartotherian
- Tracking down conflicting versions + NPM
Community Tech
[edit]- Blocked by:
- Blocking:
- Updates:
Contributors
[edit]Editing
[edit]- Blocked by:
- Blocking:
- Updates:
Parsing
[edit]- Blocked by: Services (because of Cassandra3 migration) on deployment of HTML version bump (for section, figure-inline, html5-id, interwiki-link changes). This is just an FYI since Services & us have already synced up about this.
- Blocking: Reading Infra on section tag markup
- Updates:
- itwiki, dewiki, and 170 small wikis got switched from Tidy to RemexHtml yesterday - itwiki have flagged a bunch of new issues that hadn't been caught by linting so far. Investigation ongoing and might introduce one or two newer categories to aid editors -- we expect these categories to be sparsely populated
- Templatedata related fixes being deployed today -- For pre-existing transclusions, this will prevent Parsoid from normalizing parameter order to templatedata format
Global Collaboration
[edit]- Blocked by: MW core (or Parsing?) for code review on https://gerrit.wikimedia.org/r/#/c/392990/ and surrounding stack of commits
- Blocking: ops for Flow dumps
- Updates:
- New tags for https://phabricator.wikimedia.org/T167656 are on this week's train
UI Standardization
[edit]- No OOUI release this and upcoming weeks
- Ongoing:
- OOUI & based products:
- icons: Work on icon set to be more harmonious and align to WikimediaUI Style Guide https://phabricator.wikimedia.org/T177432 finishing up
- Unify SVG markup across Foundation products https://phabricator.wikimedia.org/T178867
- Continuous work and per-project SVGO based optimizations, also initiated https://www.mediawiki.org/wiki/Manual:Coding_conventions/SVG
- OOUI & based products:
Technology
[edit]Analytics
[edit]- Blocked by:
- Blocking:
- Updates:
- The prometheus druid exporter (to report druid metrics to graphana via prometheus) got some attention and others (outside WMF) will be using it
- First productionized version of EventLogging backend on hadoop launched this week. This means SQL-friendly tables on hadoop with eventlogging data, and for the first time, ability to join to all mediawiki databases in one query
- Wikistats 2 APIs (like Pageview API, but for edit data), alpha launch: https://wikitech.wikimedia.org/wiki/Analytics/AQS/Wikistats
- Wikistats 2 UI, alpha launch: https://stats.wikimedia.org/v2/
- Decommissioning old DB hosts for EventLogging done, remember that eventlogging DB is now only on analytics-slave, not on analytics-store anymore
- Work on kafka jumbo cluster
- Rebooting one druid host for maintenance brought up unexpected issues with druid and zookeeper, working on that
Cloud Services
[edit]- Blocked by:
- Blocking:
- Updates:
- Annual Toolforge survey closed. 141 responses received (11% response rate).
- Team in Austin this week doing an offsite + attending KubeCon conference
- Will upgrade labpuppetmaster* to v4 puppet packages after the offsite
- Presuming everything in eqiad goes well next week
Fundraising Tech
[edit]- Blocked by: None
- Blocking: None
- Updates:
- New CiviCRM Import tool for Major Gifts
- Paying down some tech debt in Central Notice
- Fixes for fundraising vagrant role
- Continued improvements to Fundraising Dashboard
- Continuing to support FR-non-tech during the fundraiser
MediaWiki Platform
[edit]- Blocked by: N/A
- Blocking: Global Collaboration for code review on https://gerrit.wikimedia.org/r/#/c/392990/ and surrounding stack of commits
- Updates:
- MediaWiki 1.30 Release Candidate 0 released
- MCR and Actor table reviews ongoing
- Comment table: Tried to enable WRITE_BOTH mode on testwikis, but had to revert because it broke CentralAuth and ForeignDBViaLBRepo.
- Will likely wait until the schema change is done everywhere before we move anything to WRITE_BOTH.
- TemplateStyles: ParserOutput stateless transforms got merged. Some Wikibase issues that are being dealt with.
- Firejail: will be deployed for Score
- New PSR-4 autoloader patchset https://gerrit.wikimedia.org/r/373626 ready for review
- PoolCounter for ORES client https://gerrit.wikimedia.org/r/394407 ready for testing
- MP3 transcoding deployment
- Discussions with third party MediaWiki developers regarding extension deployment/dependency management (composer)
Performance
[edit]- Blocked by: N/A
- Blocking: N/A
- Updates:
- Significant new performance testing documentation in progress (aimed at developers), expected to be delivered before end of 2017
- New varnish slow log configuration in final testing
- perf testing of cross-data center database writes
- Q3 Goals planning: if you think you might need anything from the perf team, hit me up.
- docker-based MW dev environment (in collab with WMDE)
Release Engineering
[edit]- Blocking:
- Scoring platform, https://phabricator.wikimedia.org/T181661
- New scap release planned for this week that will hopefully address this issue.
- Scoring platform, https://phabricator.wikimedia.org/T181661
- Blocked by:
- none
- Updates:
- [MW Train] Reminder! This is your last few weeks of deployments for the year/quarter! No non-emergency deploys starts the week of December 18th.
- #mw-1.30-release MediaWiki 1.30 is about to be released in the next few days, so last call for any blockers people might have...you know...forgotten
Research
[edit]- Blocked by:
- Blocking:
- Updates:
Scoring Platform
[edit]- Blocked by:
- Parallel scap SSH issue https://phabricator.wikimedia.org/T181661
- Blocking:
- Updates:
- Outage on Nov 28, incident report is going out today.
- SimpleWiki deployed to ORES
- FYI: Global Collab, please announce and update RC Filters. The extension is only configured on the beta cluster.
- testwiki (RevIdScorer) includes thresholds now, which makes extension development easier, against the real server or in either of our vagrant roles.
- ORES Ext. Refactoring
- Incoming basic support of Icelandic (No RC Filters yet)
Search Platform
[edit]- Blocked by:
- Blocking:
- Updates:
- Working on improvements to LTR training
- Reindexed wikis to enable improved katakana/hiragana mapping https://phabricator.wikimedia.org/T179945
- Improving completion suggester interaction with namespaced prefix search https://phabricator.wikimedia.org/T178474
- Wikidata descriptions indexed, working on fulltext search
- Working on porting Selenium tests from Ruby to JS
- Logstash upgrade to ElasticSearch 5.5 on Thu, completing the 5.5 upgrade
Security
[edit]- Blocked by:
- Blocking:
- Updates:
- Lots of development on Phan security plugin by Bawolff
- Maintanenace work on security alert configuration (Github, NSP)
- Reviews:
- Ex:WikipediaExtracts
- Next set of security reviews will be scheduled this week
Services
[edit]- Blocked by: none
- Blocking: Parsing till early next week
- Updates:
- Round of Cassandra 3 bootstraps is complete
- htmlCacheUpdate for wiktionaries are on kafka queue
Technical Operations
[edit]- Blocked by:
- Global Collaboration on Flow dumps https://phabricator.wikimedia.org/T172025
- Blocking:
- None
- Updates:
- Q3 goal planning started, if teams have dependencies on Ops for next quarter, reach out
- Part of DC ops in Singapore for eqsin (new caching DC's name) rollout
- s8 (wikidata) work ongoing, work ontrack.
- Operations DBAs s8 master switchover programmed for 9th January
Wikidata
[edit]- Finally got rid of the manual "Wikidata" build process, Wikidata is part of the weekly deployment now: https://phabricator.wikimedia.org/T173818
- Introduced a custom, "compact" entity diff serialization in the wb_changes table. Please look out for side-effects: https://phabricator.wikimedia.org/T113468
- First patches demo the possibility to persist statements on sub-entities utilizing Wikibase's standard wbsetclaim API (namely Forms on Lexemes): https://phabricator.wikimedia.org/T163724
- Core changed the way it handles section editing. Wikibase hooks into this feature, and now behaves unexpected. Review welcome: https://phabricator.wikimedia.org/T181807
German Technical Wishlist
[edit]- Blocked by:
- Blocking:
- Updates: