Wikimedia Release Engineering Team/Checkin archive/2021-03-03

2020-03-03

Vacations/Important dates

https://office.wikimedia.org/wiki/HR_Corner/Holiday_List

How to do it

• 29 Mar: US staff with reqs

• 12 Apr: US staff with reqs
• 22 Apr: Earth Day -- US staff with reqs

• I made this: https://wikitech.wikimedia.org/wiki/Deployments/Yearly_calendar

Train

Maniphest query for deployment blocker tasks: https://phabricator.wikimedia.org/maniphest/query/s3KW8bpsXhYF/#R

https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Important_dates

• 16 Nov - wmf.18 - Ahmon + Antoine
• 23 Nov - wmf.19 - No Train - Thanksgiving Thurs/Fri https://phabricator.wikimedia.org/T263185
• 30 Nov - wmf.20 - Antoine + Mukunda
• 7 Dec - wmf.21 - Mukunda + Dan
• 14 Dec - wmf.22 - Dan + Jeena
• 21 Dec - wmf.23 - No Train
• 28 Dec - wmf.24 - No Train
• 4 Jan - wmf.25 - Jeena + Lars Antoine
  • NB: Lars is only back from holiday on Thursday Jan 7
• 11 Jan - wmf.26 - Lars + Jeena
• 18 Jan - wmf.27 - Brennen + Lars (Monday is a holiday)
• 25 Jan - wmf.28 - Ahmon + Brennen
• 1 Feb - wmf.29 - Antoine + Ahmon
• 8 Feb - wmf.30 - Mukunda + Antoine
• 15 Feb - wmf.31 - Dan + Mukunda (Monday is a holiday)
• 22 Feb - wmf.31 - Jeena + Dan
• 1 Mar - wmf.31 - Lars + Jeena
• 8 Mar - wmf.31 - Brennen + Lars
• 15 Mar - wmf.32 - Ahmon + Brennen
• 22 Mar - wmf.33 - Antoine + Ahmon

Status

https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Roles#Train_Conductor

SoS

• 2019-08-14 onwards: Zeljko 🎸 🎷 \o/
• 2020-08-26 onwards: Deb is in charge/SoS is async
• 2020-11-25: Brennen
• 2020-12-02: Ahmon
• 2020-12-09: Tyler
• 2020-12-16: Antoine
• 2021-01-06: Tyler
• 2021-01-13: Text only update
• 2021-01-20: Mukunda
• 2021-01-27: Text only update
• 2021-02-03: Thcipriani
• 2021-02-10: Thcipriani
• 2021-02-24: Thcipriani

Outgoing

• Blocked by:
  • PHP restart sudo rules for docroot deploy https://gerrit.wikimedia.org/r/c/operations/puppet/+/666309
• Blocking:
• Thanks
  • Cole White and O11y for all the help with the Phatality kibana extension!
• Updates:
  • [All] Deployments/Covid-19 https://wikitech.wikimedia.org/wiki/Deployments/Covid-19
  • Train Health
    • Last week: 1.36.0-wmf.32 phab:T274936
    • This week: 1.36.0-wmf.33 phab:T274937
    • Last week: 1.36.0-wmf.34 phab:T274938

Team Business

Incoming/Needs attention

• Rollback unfinished train deploys over the weekend/holidays https://phabricator.wikimedia.org/T260401
  • Backporters must be aware of state, but let's use our judgement

Book club/Lunch and Learn

• https://www.mediawiki.org/wiki/Wikimedia_Engineering_Productivity_Team/Book_club
• https://www.mediawiki.org/wiki/Wikimedia_Engineering_Productivity_Team/Lunch_and_learn
• https://www.mediawiki.org/wiki/Wikimedia_Engineering_Productivity_Team/Read_papers_and_talk
  • Feb 15th Mar 1st: Lars -- David Allen's Getting Things Done (GTD)
    • http://git.liw.fi/wmf-talks/tree/gtd.md
  • March 15th: Jeena Suggestion: Falling Down: A guide
  • Brennen suggestion (maybe): nebulous Zettelkasten rant what is this?!
    • Zettelkasten is a note taking system

Monthly reflection on accomplishments - Feb '21 edition

https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Monthly_notable_accomplishments

Add as you have them!

• PipelineLib fully working on releases-jenkins.wikimedia.org
• Rust introduction talk (not strictly RelEng business)
• logspam-watch
  • Minimum hits consolidation feature
  • Error histograms, at-a-glance status indicators (emoji, it's emoji), improved UTF-8 handling and terminal resizing
• Gearman plugin deployed. Merged bunch of pending changes + a fork from GoodData company which adds support for Pipeline jobs

Monthly reflection on accomplishments - Mar '21 edition

https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Monthly_notable_accomplishments

Add as you have them!

• PipelineLib fully working on releases-jenkins.wikimedia.org

Standup!

Ahmon

• Updates:
  • Working with Dan and others on changes to operations/mediawiki-config, etc to support offline L10N file builds
  • train-dev is broken.  Trying to revive
• Blocked by:
  • No blockers. 
• Blocking:
  •  

Antoine

• Updates:
  • Polished Kibana dashboard for Gerrit https://logstash.wikimedia.org/app/dashboards#/view/AW1f-0k0ZKA7RpirlnKV
  • Observability is there to assist in ingesting logs and mangle them to fit in ELK :]
  • Writing a large blog post about Java out of memory error / Gerrit jetty session leak. Hoping to send it for review soon.
  • gpg key signed by Tyler/Lars ( pgp-public-keys.git).  Should we do an org wide pgp signing party? (wanna read your key?)
• Blocked by:
  • Fix doc.wikimedia.org stall PHP opcache. https://gerrit.wikimedia.org/r/c/operations/puppet/+/666309
  • And can use a review on some scap config: https://gerrit.wikimedia.org/r/c/integration/docroot/+/666308 (intent is to restart php7.0-fpm after deployment)
• Blocking:
  • Bunch of people here and there

Brennen

• Blocked by:
  • It is hard for us to do things.
• Blocking:
• Updates:
  • Helping as able with GitLab bureaucracy
  • Speed & Function contractors now have access to gitlab-test.wmcloud.org
  • https://gerrit.wikimedia.org/r/c/operations/puppet/+/667310
    • logspam-watch now contains ☺️ 😐 😑 😾 😿 😎
    • Also tiny graphs
    • It turns out terminals hate emoji
  • Need to debug an fr-tech image build

Dan

• Blocked by:
  • Various mediawiki/core patches to help with offline l10n bootstrapping
    • https://gerrit.wikimedia.org/r/c/mediawiki/core/+/667977
    • https://gerrit.wikimedia.org/r/c/mediawiki/core/+/667921
    • https://gerrit.wikimedia.org/r/c/mediawiki/core/+/667906
    • https://gerrit.wikimedia.org/r/c/mediawiki/core/+/667735
  • Ahmon's changes to mediawiki-config getting review/merged
• Blocking:
  • ?
• Updates:
  • The multiversion image build job is working!
  • l10n bootstrapping works in the docker container with the aforementioned core patches applied and Ahmon's additions to mediawiki-config

Jeena

• Blocked by:
  •  
• Blocking:
• Updates:
  • If I smash my spacebar hard enough will it work normally again? The answer is YES (possible anger issues)
    • resolved: working as intended :<
  • I merged some code to mediawiki! \o/  WELL DONE! (name may have already been here: https://en.wikipedia.org/wiki/Special:Version/Credits but you're here now too :))
  • Did some reviews of mediawiki-cli patches
  • continuing work on pipeline credentials 
  • made patchset to rsync privatesettings to releases server https://gerrit.wikimedia.org/r/c/operations/puppet/+/667747

Lars

• Blocked by:
  • feelings of incompetence
• Blocking:
  • progress
• Updates:
  • train is not a disaster, but could be better

Mukunda

• Blocked by:
  • Nothing
• Blocking:
  • Nothing AFAIK
• Updates:
  • New Deployment Servers.
    •   The process of migrating to a new deployment server is still awkward dispite much effort to improve the situation.
  • Phatality is back, with much credit due to Cole White (shdubsh) \o/
    • Deployment is apparently possible now
    • Changes/improvements/regressions coming soon
  • Should be deploying some minor change to Phabricator this evening

Tyler

• Blocked by:
  • Nothing
• Blocking:
  • ???
• Updates:
  • GitLab
  • Code stewardship-ish... meetings
  • annual planning