Scrum of scrums/2016-11-23

2016-11-23

Technology

Analytics

• Blockers: none
• on track for quaterly goals
• main project about edit data (mediawiki edit history reconstruction) progressing,
  • we are now calculating standard edit metrics for all wikis since the beginning of time using denormalized edit history: https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Denormalized_edit_history experimental dataset on pivot

• working on productionizing infrasructure for event streams
• waiting for hardware for pageview API
• owning now statsv together with ops (utility that can consume kafka data and report to graphana)
• Thanking discovery for contributing to our metric reporting tools

Upcoming:

• Start design work to revamp information architecture of http://stats.wikimedia.org

Performance

Not blocking, not blocked

• thanks everyone who attended the active/active DC meeting after I flagged it here, it has helped getting the ball rolling on two blockers
• hidden tabs confirmed as messing with timing data, now excluded from perf metrics
• investigating little-known legacy features in mediawiki thumbnailing to decide whether we continue supporting them on Thumbor (302 redirects)
• second view tests added for firefox and IE in WebPageTest (was previously only looking at Chrome)
• still active on thumbnail URL/API RFC discussion
• briefly discussed witth multimedia team setting up Thumbor for them to leverage in their ImageTweaks extension

Security

• Security Reviews
  • Linter review complete
  • LoginNotify schedule for this week

• Continuing work on wiki account compromise remediation (T150554)
  • Assistance needed -- e-mail to engineering@ is forthcoming with request

Services

• Blockers: none
• Updates:
  • PDF render service deployed in codfw, eqiad and public exposure next week
  • New version of service-template-node: ES6 and ESLint are coming

Technical Operations

• Blockers
  • IOS native app
    • Requesting timeline for Wikipedia iOS app requesting 0px thumbs: https://phabricator.wikimedia.org/T147648 https://phabricator.wikimedia.org/T151078
    • iOS 5.3.0 was shipped last week
      • Performance ?
    • MW fix to return 400 on 0px thumbs https://phabricator.wikimedia.org/T147784
• Blocking
  • None

• Updates
  • jobqueue woes https://phabricator.wikimedia.org/T151196
  • kubernetes/calico work ongoing, goal on track
  • dropping varnish 3 compatibility code from our puppet repos
  • labsdb goals on track as well

Release Engineering

• Blocking
• Blocked
• Updates
  • Mediawiki 1.28 tarball release this week!

Product

Reading

Mobile Content Service (MCS)

• Board: https://phabricator.wikimedia.org/tag/mobile-content-service/
• Added announcements feed endpoint (public now). More info and request URL at https://www.mediawiki.org/wiki/Wikimedia_Apps/Team/RESTBase_services_for_apps#....2Ffeed.2Fannouncements:

Android native app

• Last week:
  • Continuing Q2 goals for Wikidata descriptions
  • New fundraising announcement explore feed card in progress
  • Now building against Android Nougat 7.1 API 25
  • Fixing login and editing issues
  • Lots of unit tests
• Next week (https://phabricator.wikimedia.org/project/view/2352/):
  • More Q2 goals for Wikidata descriptions (tutorial and polish)

iOS Native App

• Last Week:
  • Shipped 5.3.0 (In the news notifications & feed content, MCS backed feed, language variant support, other bug fixes and enhancements) https://phabricator.wikimedia.org/project/view/2220/
  • Added announcement card to the feed (for user research and fundraising)
  • Started update of data layer to fix issue with data access & modification from widgets and notifications
  • Started dynamic font size updates to the app
• This week: https://phabricator.wikimedia.org/project/view/2357/
  • Finishing data layer update
  • Continuing dynamic font size updates
  • Other minor bug fixes for 5.3.1

Web

• Current sprint:https://phabricator.wikimedia.org/tag/reading-web-sprint-86-
  • Stopping HoverCards A/B tests from Russian and Italian wikis
  • New readers work
  • Make PageImages return the image in the lead section
  • MobileFrontend tech debt
  • Trending service
  • Hovercards rewrite
• Next week: probably the same stuff as the current week.

Reading Infrastructure

• not blocking
• Still looking for reviews on the API error/warning i18n patches
  • https://gerrit.wikimedia.org/r/#/c/321402/ - Improve handling of Message objects as Message parameters
  • https://gerrit.wikimedia.org/r/#/c/321404/ - Add Message::listParam()
  • https://gerrit.wikimedia.org/r/#/c/321405/ - Fix MediaTransformError message handling
  • https://gerrit.wikimedia.org/r/#/c/321406/ - API: i18n for warnings and errors
  • The extension patches depended on by that change are next in importance. These are for OAuth, TitleBlacklist, GlobalBlocking, Translate, and ConfirmEdit.
  • All other WMF-deployed extensions affected by this change have patches too, see https://gerrit.wikimedia.org/r/#/q/topic:api-error-i18n/T47843. Non-WMF-deployed extensions are (mostly) not touched at this time, the worst that should happen to them is wfDeprecated warnings eventually.
• https://gerrit.wikimedia.org/r/#/c/312865/ is blocked on review by Security

Community Tech

• Not blocking
• Blocker: Need a security review for https://phabricator.wikimedia.org/T150832 to proceed with exposing a couple of table views on tool labs db
• Community Wishlist survey proposal phase over. https://meta.wikimedia.org/wiki/2016_Community_Wishlist_Survey
• Did bug-fixing for Copypatrol (plagiarism detection tool) and launched French version: https://tools.wmflabs.org/copypatrol/fr
• Ongoing RfC about changing default collation on Meta: https://meta.wikimedia.org/wiki/Requests_for_comment/Switch_default_category_collation_to_UCA_collation_with_numeric_sorting
• Ongoing RfC about abandoned labs tools takeover: https://meta.wikimedia.org/wiki/Requests_for_comment/Abandoned_Labs_tools
• Ongoing work with programs dashboard

Discovery

• BM25 scoring enabled on 10 larges wikis
• Discovery mission & roadmap presentation: https://docs.google.com/presentation/d/1ctlqdLA__0OxDuO7mJEIDLP-xt9a7E4jv4INMlZAHdQ
• Load-testing crosswiki searching backend code
• Portal updates per-language article count stats, dewiki joins the lucrative 2M+ club :)

Editing

Language

• Blocked: T150512: WikiBase Repo tests failing with UsageException
  • This is making it difficult to merge Translate patches. Issue seems to be in database clearing in tests. QA/RelEng?

Collaboration

• Blocked: None
• Blocking: None
• Updates
  • No deployments this week. Ongoing work on:
    • Mobile support for left nav of Special:Notifications
    • RecentChanges filters and filter framework for Edit Review Improvements

Fundraising Tech

• Big English fundraiser starts next week!
  • repeating Greg's emailed plea: https://lists.wikimedia.org/pipermail/engineering/2016-November/000331.html
  • Stuff that could impact the fundraiser: GeoIP, ResourceLoader, MessageCache, EventLogging, Hive webrequest tables
• CentralNotice: reviewing Aaron Schultz's latest MessageCache patch: https://gerrit.wikimedia.org/r/#/c/318489
  • We want to understand it really well before we deploy anything that could affect banners
  • If anyone with deep knowledge of MessageCache (Aaron, Gilles?) has time for a quick video chat, Andrew Green has a few questions
  • As always, more scrutiny and comments are welcome.
• Deployed Nirzar's mobile CSS fixes, looking great so far
• Minor caching optimizations for CiviCRM jobs