2024-07-31

Last time

🏆 Wins/winterrogation

https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Monthly_notable_accomplishments

July 2024

"[WE6.2.1] Publish pre-train single version containers" is now on Phab <https://phabricator.wikimedia.org/T369115>
Submitted a WMCS themed talk proposal for WikiCon North America (<https://wikiconference.org/wiki/Submissions:2024/What%27s_new_with_Wikimedia_Cloud_Services>)
https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/367 kubernetes: Make k8s deployment failures fatal
https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/101 Allow empty requirements list to enable Python builder
- Cleaned up scap, kokkuri and mediawiki/services/machinetranslation
https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/368 Prompt for log message if not supplied on command line
Deployed gitlab-runner v17.0.0. on gitlab-cloud-runners.
https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1047158 mw-web: Add traindev environment — testing canary deployments should work!
https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/76 enable mw-web canary / main k8s deployment
Fixed train-dev for 1.43.0-wmf.11
Blubber v1.0.0 has been published
- Native BuildKit LLB (Low-Level Build) instructions (no more reliance on Dockerfile)
- Refactored to support all `docker build` and `docker buildx build` options
- Supports attestations stored alongside images in the registry, provenance and SBOM
Looks like phorge 2024.19 stable release merges cleanly, as does phorge/master
Settled on a squash commit template: https://gitlab.wikimedia.org/repos/releng/gitlab-settings/-/merge_requests/64
Enabled image diff in Gerrit: https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/902211/1/static/images/project-logos/dkwikimedia.png
Further refinement in upstream code would allow some more tuning, screenshots at https://phabricator.wikimedia.org/T341291#9939660
Diff is based on Resemble.js library, demo at http://rsmbl.github.io/Resemble.js/
MediaWiki train!!!! Win win!
Started building out an on-wiki space to keep track of group-1 things: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Group_-1
https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/100: Change kokkuri to use v2.0.3 (newest buildctl)
https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/107 Fix duplicate pipelines when pushing merge request
https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/377 sync: Refactor deployment stages
- thx to swfrench for reviews (nice)
Dropped logstash_checker.py from puppet (nice :))
Filed https://phabricator.wikimedia.org/T369532: Update ldap-sync-bot token
We have a director
Met with Guillaume re: archiva migration, publishing doesn't seem to be working but we'll debug that this afternoon.
Discussed gitlab-behind-loadbalancing ideas with Collab Services, they may be dissuaded from changing SSH remotes
Blubber can now build directly from Git URLs. This makes the provenance more "complete"
- Adding support for remote build contexts to Kokkuri. In the GitLab CI context this means passing the MR Git URL directly to BuildKit
Rephrased the GitLab account approval banner because confusion: https://phabricator.wikimedia.org/T369698
- Updated the GitLab approval form to remove with the "specific reason" for GitLab
Continued with writing random Phab downstream and Phorge upstream tech debt patches and bug fixes
Talked with folks who like quarterly Phab account metrics about lies, damned lies, and statistics (thanks Tyler)
https://phabricator.wikimedia.org/T369862 Upgrade to buildkit v0.15.0
ldap-sync-bot token has been renewed.
Braggedlogged about latest Phabricator improvements in https://phabricator.wikimedia.org/J316
Phab/Phorge up to date with upstream stable (next step: rolling release) oooh
invented https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Group_-1/Progress_reports/2024-07-11 to put weekly reports on-wiki
dzahn got Gerrit on nftables, no obvious breakage
Sorta got Maven publishing working at https://gitlab.wikimedia.org/repos/maven/maven-test-project
- Open question: Any reason to care about just using a single project for all kinds of packages?
- https://phabricator.wikimedia.org/T367322#9987048
Swapping out VMs for CI/Integration-project --- cumin done! deb building in progress.
Explored running old-timey patchdemo in k8s alongside catalyst
Another routine Phab/Phorge deploy
Some less dependencies upon EOL Debian Buster
Merged modifications for repos/releng/release
- (make-container-image), operations/mediawiki-config
- (MWMultiversion.php), and scap to support FORCE_MW_VERSION, to support

single image container images.

Andre's "scap train" mods deployed.
scap changes to better support alternate stage_dir:
- https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/390 cli.py: Change default path of mediawiki sync lockfile
- https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/389 Make default history_log follow stage_dir
Southparkfan hammering away at beta mess
streaming logs from Catalyst environments
It seems like the new deployment box is pretty much working by now.
- Jaime had to fix git flag placement for git 2.20 -> git 2.30
- Jaime had to fix scap deploy for heterogenous python versions within the cluster
Merged changes in catalyst for MediaWiki helm charts to spin up new MediaWiki instances with the PatchDemo provisioning scripts
Progress on merging the PatchDemos
Split Puppet 5 and 7 compiler output since some hosts no more support v5 and that was confusing SREs (screenshots: https://phabricator.wikimedia.org/T371407#10028859 )
Added a "(diff)" link to the notification that https://schedule-deployment.toolforge.org/ gives you after adding a new backport to the schedule. phab:T367948
git.wikimedia.org is finally dead (a win in so far as maybe we never have to talk about it again)
Quarterly phabricator queries updated
Upstream opengraph diff was merged for phab, so that link previews may start working in slack at some point
Adding milestone description copying to lessen suffering

Stuff from last time

📅 Vacations/Important dates

https://office.wikimedia.org/wiki/HR_Corner/Holiday_List#2024

https://wikitech.wikimedia.org/wiki/Deployments/Yearly_calendar

https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Time_off

July 1-9: Jaime
July 4-5: US staff holiday
July 4-5: Andre (festival; CZ Holiday)
July 12: Bryan
July 12–15: Ahmon
July 12: Brennen WF Las Vegas
July 12-22: Dan
July 19: Bryan
July 25-31: Ahmon
July 26: Bryan
Aug 02: Bryan
Aug 05-08: Dan
Fri 09 Aug – Global holiday: International Day of the World’s Indigenous Peoples
Aug 12: Dan
Mon 12 Aug-16: Ahmon out
Mon 12 Aug - Fri 23 Aug: Antoine
Aug 16: Bryan
Aug 23: Bryan
Aug 23: Jaime
Sun 25 Aug - 03 Sep: Brennen 🔥
Aug 30: Bryan
Sept 02: US Labor day (WMF US holiday)
Sept 06: Bryan
Sept 13: Bryan
Sept 18-19: Brennen Winfield
Sept 19-20, 23: Bryan Riot Fest in Chicago!
Sept 27: Bryan
Sept 11-30 Jeena
Oct 03-06: WikiCon North America (Indianapolis)
Oct 1-11 Jeena

Future

🔥🚂 Train

https://versions.toolforge.org/

https://train-blockers.toolforge.org/

https://wikitech.wikimedia.org/wiki/Deployments/Yearly_calendar

Rotation

13 May (13-17) - 1.43.0-wmf.5 – Antoine + Andre (Jeena as backup)
20 May (20-24) - 1.43.0-wmf.6 – Andre + Antoine (you can do it Andre!) (Antoine out Mon, Ahmon out Fri)
27 May (27-31) - 1.43.0-wmf.7 – Ahmon + Andre (Memorial day Monday)
03 Jun (03-07) - 1.43.0-wmf.8 – Dduvall + Ahmon (Brennen out)
10 Jun (10-14) - 1.43.0-wmf.9 – Brennen + Ahmon
17 Jun (17-21) – 1.43.0-wmf.10 – Jaime + Brennen
24 Jun (24-28) – 1.43.0-wmf.11 – Jeena + Jaime
01 Jul (02–04) – 1.43.0-wmf.12 – Antoine + Jeena
08 Jul (08-12) – 1.43.0-wmf.13 – Andre + Antoine (Ahmon + Dan out Friday)
15 Jul (15-19) – 1.43.0-wmf.14 – Ahmon + Andre (Dan Out)
22 Jul (22-26) – 1.43.0-wmf.15 – Dan + Ahmon (Ahmon out Thurs)
- Temp accounts is happening:
29 Jul (19-Aug 02) – 1.43.0-wmf.16 – Brennen + Dan (Ahmon out)
05 Aug (05-09) – 1.43.0-wmf.17 – Jaime + Brennen (Dan out, Global holiday Friday)
12 Aug (12–16) – 1.43.0-wmf.18 – Jeena + Jaime (Ahmon out, Antoine out)
19 Aug (19–23) – 1.43.0-wmf.19 – Andre + Jeena (Antoine out)
26 Aug (26–30) – 1.43.0-wmf.20 – Antoine + Andre (Brennen out)
02 Sep (02–06) – 1.43.0-wmf.21 – Ahmon + Antoine (US holiday Monday, Brennen out Tues)
09 Sep (09–13) – 1.43.0-wmf.22 – Dan + Ahmon
16 Sep (16–20) – 1.43.0-wmf.23 – Jaime + Dan (Brennen out)
23 Sep (23–27) – 1.43.0-wmf.24 – Brennne + Jaime

Team Discussions

Fill your day with webforms

Offsite dates: https://docs.google.com/forms/d/e/1FAIpQLSdFM8C7RaRSXYgxefMN3ak03A8Vj1v8N-1wtOjOfCMlAtap5Q/viewform

Let's do some inbox triage: https://phabricator.wikimedia.org/maniphest/query/7vRDrcVnt8OI/#R

Inbox triage decision:

GitLab approvals: if they fill out the form right, let's approve them, not add them trusted-contributors unless they actually are

🌻 Open source/Upstream contributions

https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Upstream