Wikimedia Release Engineering Team/Checkin archive/20191009
Appearance
2019-10-09
[edit]Vacations/Important dates
[edit]- October 8: Željko holiday (Independence Day)
- October 11: Željko conference (https://2019.webcampzg.org/ )
- October 14 (Indigenous Peoples' Day): US Staff
- October 25–26: James at WikidataCon 2019 in Berlin
- November 11 (Veterans' Day): US Staff
- November 12–15: TechConf19
- November 16–20: Offsite
- November 21-30: Jeena
- November 25-29: Dan
- November 28–29 (Thanksgiving): US Staff
- December 1-2: Jeena
- December 6: Lars (Finnish Independence Day)
- December 25–31 (Christmas): US Staff
- December 25–26: Lars, Christmas
- 1 January (New Year's Day): US Staff, Lars
- 20 January (Martin Luther King Jr. Day): US Staff
Rotating positions
[edit]Train
[edit]- Maniphest query for deployment blocker tasks: https://phabricator.wikimedia.org/maniphest/query/s3KW8bpsXhYF/#R
- Jun 24 - wmf.11 - Jeena (with Mukunda)
- Jul 1 - wmf.12 - No Train (Fourth of July)
- Jul 8 - wmf.13 - Jeena
- Jul 15 - wmf.14 - Lars (with Antoine)
- Jul 22 - wmf.15 - Lars
- Jul 29 - wmf.16 - Brennen (with Tyler)
- Aug 5 - wmf.17 - Brennen
- Aug 12 - wmf.18 - No Train (Wikimania)
- Aug 19 - wmf.19 - Zeljko 😱
- Aug 26 - wmf.20 - Zeljko 😭
- Sep 2 - wmf.21 - Antoine
- Sep 9 - wmf.22 - Antoine
- Sep 16 - wmf.23 - Mukunda
- Sep 23 - wmf.24 - Mukunda
- Sep 30 - wmf.25 - Tyler Dan
- 7 Oct - 1.35.0-wmf.1 - Tyler Dan
- 17 Oct - wmf.2 - Jeena + Lars
- 21 Oct - wmf.3 - Lars + Brennen
- 28 Oct - wmf.4 - Brennen + Mukunda
- 4 Nov - wmf.5 - Mukunda + Antoine
- 11 Nov - wmf.6 - No Train (Tech conf)
- 18 Nov - wmf.7 - No Train (Off site)
- 25 Nov - wmf.8 - Antoine + Jeena (Thanksgiving)
- 2 Dec - wmf.9 -
- 9 Dec - wmf.10 -
- 16 Dec - wmf.11 -
- 23 Dec - wmf.12 - No Train (Winterval)
- 30 Dec - wmf.13 - No Train (Still Winterval)
- 6 Jan - wmf.14 -
- 13 Jan - wmf.15 -
TODO thcipriani to figure out thanksgiving wit hfolks
- antoine ok for thursday, can cut branch/sec patch on tuesday. Rest depend on jetlag of doom :-\
SoS
[edit]- Aug 14 onward: Zeljko
- SoS on 2019-10-09 is at the same time as Tech All Hands
Team Business
[edit]Timespent spreadsheet
[edit]- For the avoidance of doubt: fill out the sheet week number for the previous week
- spreadsheet for week 40 (2019-09-30): https://docs.google.com/spreadsheets/d/1YS_2eUQWUOnygshMfxgGPu93GKGCO5t00yjUREQ5B-I/edit#gid=0
- Ping Antoine, Brennen, Greg, Jeena, Lars
Clinic Duty Proposal
[edit]November Offsite/Summit
[edit]- TechConf19: November 12–15
- Offsite: November 16–20
- Arrive the 15th
- 16th is our fun day/outing
- 17–20 are working days
- 21st fly home
- https://etherpad.wikimedia.org/p/EngProdTeam-Offsite-2019-11-planning
Book club
[edit]- https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Book_club
- 2019-10-15
- Read the blog post: http://blog.codepipes.com/testing/software-testing-antipatterns.html
- Let's discuss! https://etherpad.wikimedia.org/p/releng-book-club
- 2019-10-15
Monthly reflection on accomplishments - September '19 edition
[edit]- https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Monthly_notable_accomplishments
- Add as you have them!
- Scap 3.12.1-1 released/deployed
- Refactored Zuul layout to use per-branch pipelines
quibble -c
Lets you run arbitrary code against a working MediaWiki install- The phabricator "Report Error Code" form (https://phabricator.wikimedia.org/maniphest/task/edit/form/46/ ) has been updated with separate fields for the stack trace and error code/request id.
- T232608 Delete selenium-daily-beta-EXTENSION Jenkins jobs that are broken more than 30 days
- Write cached config to JSON as well as serialised PHP https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/533592/ (first step towards a saner config)
- MediaWiki PHP support target modernised from 7.0+ to 7.2+ for 1.34 onwards. https://phabricator.wikimedia.org/T228342
- Quibble 0.0.35 release
- 1.34.0-wmf.24 branch cut was done /mostly/ with branch.py instead of make-wmf-branch.php (some small bugs remain to work out but it's very close)
- Creating accounts was broken on beta cluster since 2019-09-08. It was fixed today (2019-09-25). https://phabricator.wikimedia.org/T232796
- Phatality extension for Kibana deployed to production and used for reporting production errors into Phabricator.
- Train blocker tasks created for 1.35.0-wmf.1—1.35.0-wmf.25
- Dev images are now automatically created as part of postmerge via the pipeline for MediaWiki
September copied to https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Monthly_notable_accomplishments#September as of 2019-10-09 17:16 UTC
October
[edit]- Dev images are now automatically created as part of postmerge via the pipeline for:
- Parsoid
- Soon: RestBASE
- Selenium documentation updated https://www.mediawiki.org/wiki/Selenium/Node.js
- Quibble 0.0.36 released https://lists.wikimedia.org/pipermail/wikitech-l/2019-October/092658.html
- Quibble 0.0.37 released https://lists.wikimedia.org/pipermail/wikitech-l/2019-October/092660.html
- Quibble 0.0.38 released
- Introducing Phatality - Streamlined error reporting from Kibana to Phabricator https://phabricator.wikimedia.org/phame/post/view/177/introducing_phatality/
- HHVM removed from CI and MediaWiki.
Incoming/Needs attention
[edit]Scrum of Scrums for next week
[edit]Internal team notes, not for pasting into the SoS notes
[edit]Incoming from last week (aka this morning)
[edit]Outgoing next week
[edit]- Blocked:
- Blocking:
- Updates:
- Train Health
- Last week: 1.35.0-wmf.1 - phab:T233849
- This week: 1.35.0-wmf.2 - phab:T233850
- Next week: 1.35.0-wmf.3 - phab:T233851
- Introducing Phatality - Streamlined error reporting from Kibana to Phabricator https://phabricator.wikimedia.org/phame/post/view/177/introducing_phatality/
- Quibble 0.0.36 released https://lists.wikimedia.org/pipermail/wikitech-l/2019-October/092658.html
- Quibble 0.0.37 released https://lists.wikimedia.org/pipermail/wikitech-l/2019-October/092660.html
- Selenium documentation updated https://www.mediawiki.org/wiki/Selenium/Node.js
- Train Health
Callouts
[edit]Train status and happenings
[edit]OKRs, Goals, Betterworks, Oh my!
[edit]- Concerns? Questions no one can answer?
- Additionally (yes, that's right)...Team Goals
Standup!
[edit]- Make sure your tasks are correct on our kanban board!
Antoine
[edit]- What I planned vs did last week
- Tech conf sessions reviews. Gotta lead one or even two of them?
- Did Quibble reviews and a few faulty releases :-\
- Fall into a Gerrit trap
- Bitergia bot taking too much Gerrit resources, to be relocated to gerrit-replica (might have been done already)
- What I plan to do this week
- (LATE) More extraction of Selenium to standalone jobs ( https://phabricator.wikimedia.org/T232759 )
- Need full integration test for Quibble (namely actually run it on each patchset proposals)
- Stretch VM are slower than the Jessie ones https://phabricator.wikimedia.org/T232706
- Gotta check with WMCS team to benchmark that on their idling test cloudvirt hosts
- What I'm blocked on
- Some cloudvirt are too slow (T225713 / T223971), needs BIOS change.
- contint1001 has lot of dangling Docker images. Cleaned 47GBytes of them today (Oct 9th)
- For Gerrit we need a JMX Prometheus exporter to monitor the JVM https://phabricator.wikimedia.org/T184086
- Other?
- Logstash on beta is still broken https://phabricator.wikimedia.org/T233134
Brennen
[edit]- What I planned vs did last week
- Done Pair with Jeena on generating some service dev images in pipeline
- parsoid's working now
- mediawiki: Done T234379 MediaWiki pipeline config: Correctly tag development images with dev
- restbase should follow shortly (hopefully?)
- Done Read for book club
- In progress Dev environment experimentation
- In progress T234391 Point deployment-charts/mediawiki-dev at latest dev image published by pipeline
- Not done Put OKRs in Betterworks
- Not done Figure out what violence Enigmail and Keybase did to my GPG config
- Unplanned:
- Feedback on job posting
- Mail to wikitech-l about dev image generation
- Done Pair with Jeena on generating some service dev images in pipeline
- What I plan to do this week
- Merge deployment-charts/mediawiki-dev
- OKRs / Betterworks
- GPG
- Dev environment hackery
- Input/voting on tech conf sessions
- Familiarize myself with Seakeeper proposal
- What I'm blocked on
- Other?
- Closing on house tomorrow at 10:30am local, out for a bit
Dan
[edit]- What I planned vs did last week
- Done 1.34.0-wmf.25 train
- Done Branched REL1_34 🎉
- Ran into some issues here with the existing scripts and process
- Spent some time overhauling the scripts.
- Thanks for the help, Mukunda and James!
- Hopefully it will go more smoothly for the next sucker. 'tis one command.
- In progress Worked on Seakeeper K8s Proposal
- What I plan to do this week
- 1.35.0-wmf.1 deployment blockers
- Deployment Pipeline fails with CPS error for Kartotherian
- Continue to flesh out Seakeeper K8s proposal
- Areas of specific focus to improve are: estimation of resource needs and delineation of access control (namespaces of subsytems, project workflows, and ideally who should have access)
- What I'm blocked on
- Other?
Greg
[edit]- What I planned vs did last week
- TechConf - mostly on track
- What I plan to do this week
- OKRs
- TechConf scheduling
- Please do review sessions and provide input
- Also, see the emails I sent to the releng@ list about these, seeking session leads
- Pipeline
- CI futures
- Airtable sense-making (probably will be back-burnered)
- What I'm blocked on
- Other?
James
[edit]- What I planned vs did last week
- Done Supporting 1.34 release branch cut/etc.
- Done OKR fun.
- Done HHVM removal from CI and later MediaWiki
- In progress Static configuration progress https://phabricator.wikimedia.org/T223602
- In progress Iterating from CodeReview metrics/KPI proposal to make some for CI
- A bit of unplanned support for SDC.
- What I plan to do this week
- More static configuration progress
- PHP72 / PHPUnit8 work continues \o/
- 1.34.0-rc.0 support
- TechConf session scope/direction input
- UBNs as they arise.
- What I'm blocked on
- Input from Antoine on how to test https://gerrit.wikimedia.org/r/c/integration/config/+/539987 for https://phabricator.wikimedia.org/T234062
- Input from Antoine on how to fix https://gerrit.wikimedia.org/r/c/integration/config/+/540683 for https://phabricator.wikimedia.org/T230446
- (Antoine): I am late on my integration/config reviews (due to Gerrit/Quibble business)
- No worries. :-)
- (Antoine): I am late on my integration/config reviews (due to Gerrit/Quibble business)
- Still seeking input from deployers/configurators on plans for static configuration changes.
- Other?
Jean-Rene
[edit]- What I planned vs did last week
- What I plan to do this week
- What I'm blocked on
- Other?
Jeena
[edit]- What I planned vs did last week
- Done make example of deployment charts template problem for Alexandros
- In progress OKR writing
- Done local dev meeting & prep
- Done Work with Brennen to get parsoid image built in CI. In progress Same for restbase? Good question.
- Not done Explore what's needed for new php parsoid service in terms of local-charts/deployment-charts
- Done Reading for book club
- mediawiki image benchmarking
- What I plan to do this week
- meetings
- timespent spreadsheet
- Explore what's needed for new php parsoid service in terms of local-charts/deployment-charts
- OKRs
- work with zeljko on getting local-charts running
- book club
- tech conf voting
- look at seakeeper proposal
- work on some other local dev poc
- What I'm blocked on
- Other?
Lars
[edit]- What I planned vs did last week
- Not done Seek feedback on v3 of CI arch doc.
- Not done Set up Argo in a K8s cluster somewhere.
- Not done Update engprod list on PGP keysigning at offsite.
- Not done Read the book club article. Let subconsicous start work on it.
- Got over my K8s issue, helped by Jeena.
- Gave a PGP basics tutorial, helped dvpita to generate key.
- What I plan to do this week
- Seek feedback on v3 of CI arch doc, after minor updates.
- Set up Argo in a K8s cluster somewhere.
- Update engprod list on PGP keysigning at offsite.
- Book club.
- Talk with people going to TechConf about new CI.
- What I'm blocked on
- Other?
- not being well this week
- away Thursday evening Finnish time for Rust meetup
Mukunda
[edit]- What I planned vs did last week
- Done Branch REL1_34 with Dan
- Done Work on branch.py (monthly achievements!)
- Not done Phabricator Deploy
- Not done Work on getting phabricator to notify you when attempting to submit a duplicate phatality report.
- What I plan to do this week
- Phatality Deployment
- Phabricator Deployment
- Work on Phabricator migrations and disaster recovery stuff
- Come up with some topics for the focused patch triage (action item for code review working group)
- What I'm blocked on
- Other?
Tyler
[edit]- What I planned vs did last week
- Done Leveling
- Done OKRs
- What I plan to do this week
- connect with SRE re New CI
- Set date for Gerrit move to new server
- OKR...fiddling
- What I'm blocked on
- Other?
Zeljko
[edit]- What I planned vs did last week
- Done T213268 Upgrade webdriverio to version 5 in mediawiki/core - by Timo, I was a reviewer
- In progress T234002 Make MediaWiki Wdio tests less slow (Sept 2019)
- In progress T199113 All repositories with Selenium tests should use wdio-mediawiki
- Done T234311 Fix and restore daily browser tests for TwoColConflict
- Not done T232638 Add accessibility check for Homepage
- What I plan to do this week
- Stare at SafeGuard Letter. For hours. Then, sign it.
- T199113 All repositories with Selenium tests should use wdio-mediawiki
- T234610 ERROR webdriver: Request failed due to Error: session not created: Chrome version must be between 70 and 73
- T232638 Add accessibility check for Homepage
- T234085 WMTC19: Themes/Focus Areas Proposals
- T234088 Theme: Testing
- T234635 Wikimedia Technical Conference 2019 Session: System level testing: patterns and anti-patterns with Selenium
- T234088 Theme: Testing
- What I'm blocked on
- I can't reproduce CI failure on my machine, making it hard to fix :/ https://gerrit.wikimedia.org/r/c/mediawiki/extensions/TemplateWizard/+/540894#message-fa48dda448cd609a6ca513c807ee3e47aaa9b827
- Other?
- Not blocked but stuck: T233960 Error: error installing: the server could not find the requested resource
- At a local conference on Friday
- Running a half maratnon on Sunday. I've run several, but the last one was 12 years ago.