Wikimedia Release Engineering Team/Checkin archive/2024-07-17
2024-07-17
[edit]🏆 Wins/winterrogation
[edit]- https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Monthly_notable_accomplishments
- July 2024
- "[WE6.2.1] Publish pre-train single version containers" is now on Phab <https://phabricator.wikimedia.org/T369115>
- Submitted a WMCS themed talk proposal for WikiCon North America (<https://wikiconference.org/wiki/Submissions:2024/What%27s_new_with_Wikimedia_Cloud_Services>)
- https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/367 kubernetes: Make k8s deployment failures fatal
- https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/101 Allow empty requirements list to enable Python builder
- Cleaned up scap, kokkuri and mediawiki/services/machinetranslation
- https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/368 Prompt for log message if not supplied on command line
- Deployed gitlab-runner v17.0.0. on gitlab-cloud-runners.
- https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1047158 mw-web: Add traindev environment — testing canary deployments should work!
- https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/76 enable mw-web canary / main k8s deployment
- Fixed train-dev for 1.43.0-wmf.11
- Blubber v1.0.0 has been published
- Native BuildKit LLB (Low-Level Build) instructions (no more reliance on Dockerfile)
- Refactored to support all `docker build` and `docker buildx build` options
- Supports attestations stored alongside images in the registry, provenance and SBOM
- Looks like phorge 2024.19 stable release merges cleanly, as does phorge/master
- Settled on a squash commit template: https://gitlab.wikimedia.org/repos/releng/gitlab-settings/-/merge_requests/64
- Enabled image diff in Gerrit: https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/902211/1/static/images/project-logos/dkwikimedia.png
- Further refinement in upstream code would allow some more tuning, screenshots at https://phabricator.wikimedia.org/T341291#9939660
- Diff is based on Resemble.js library, demo at http://rsmbl.github.io/Resemble.js/
- MediaWiki train!!!! Win win!
- Started building out an on-wiki space to keep track of group-1 things: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Group_-1
- https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/100: Change kokkuri to use v2.0.3 (newest buildctl)
- https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/107 Fix duplicate pipelines when pushing merge request
- https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/377 sync: Refactor deployment stages
- thx to swfrench for reviews (nice)
- Dropped logstash_checker.py from puppet (nice :))
- Filed https://phabricator.wikimedia.org/T369532: Update ldap-sync-bot token
- We have a director
- Met with Guillaume re: archiva migration, publishing doesn't seem to be working but we'll debug that this afternoon.
- Discussed gitlab-behind-loadbalancing ideas with Collab Services, they may be dissuaded from changing SSH remotes
- Blubber can now build directly from Git URLs. This makes the provenance more "complete"
- Adding support for remote build contexts to Kokkuri. In the GitLab CI context this means passing the MR Git URL directly to BuildKit
- Rephrased the GitLab account approval banner because confusion: https://phabricator.wikimedia.org/T369698
- Updated the GitLab approval form to remove with the "specific reason" for GitLab
- Continued with writing random Phab downstream and Phorge upstream tech debt patches and bug fixes
- Talked with folks who like quarterly Phab account metrics about lies, damned lies, and statistics (thanks Tyler)
- https://phabricator.wikimedia.org/T369862 Upgrade to buildkit v0.15.0
- ldap-sync-bot token has been renewed.
- Braggedlogged about latest Phabricator improvements in https://phabricator.wikimedia.org/J316
- Phab/Phorge up to date with upstream stable (next step: rolling release) oooh
- invented https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Group_-1/Progress_reports/2024-07-11 to put weekly reports on-wiki
- dzahn got Gerrit on nftables, no obvious breakage
- Sorta got Maven publishing working at https://gitlab.wikimedia.org/repos/maven/maven-test-project
- Open question: Any reason to care about just using a single project for all kinds of packages?
- https://phabricator.wikimedia.org/T367322#9987048
- Swapping out VMs for CI/Integration-project --- cumin done! deb building in progress.
- Explored running old-timey patchdemo in k8s alongside catalyst
Stuff from last time
[edit]📅 Vacations/Important dates
[edit]- https://office.wikimedia.org/wiki/HR_Corner/Holiday_List#2024
- https://wikitech.wikimedia.org/wiki/Deployments/Yearly_calendar
- https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Time_off
- July 1-9: Jaime
- July 4-5: US staff holiday
- July 4-5: Andre (festival; CZ Holiday)
- July 12: Bryan
- July 12–15: Ahmon
- July 12: Brennen WF Las Vegas
- July 12-22: Dan
- July 19: Bryan
- July 25-31: Ahmon
- July 26: Bryan
- Aug 02: Bryan
- Aug 05-09: Dan (I haven't told Tyler yet, shhh)
- Fri 09 Aug – Global holiday: International Day of the World’s Indigenous Peoples
- Mon 12 Aug-16: Ahmon out
- Aug 16: Bryan
- Aug 23: Bryan
- Sun 25 Aug - 03 Sep: Brennen 🔥
- Aug 30: Bryan
- Sept 06: Bryan
- Sept 13: Bryan
- Sept 18-19: Brennen Winfield
- Sept 19-20, 23: Bryan Riot Fest in Chicago!
- Sept 27: Bryan
- Sept??? Jeena
- Oct 03-06: WikiCon North America (Indianapolis)
Future
[edit]🔥🚂 Train
[edit]- https://versions.toolforge.org/
- https://train-blockers.toolforge.org/
- https://wikitech.wikimedia.org/wiki/Deployments/Yearly_calendar
Rotation
[edit]- 13 May (13-17) - 1.43.0-wmf.5 – Antoine + Andre (Jeena as backup)
- 20 May (20-24) - 1.43.0-wmf.6 – Andre + Antoine (you can do it Andre!) (Antoine out Mon, Ahmon out Fri)
- 27 May (27-31) - 1.43.0-wmf.7 – Ahmon + Andre (Memorial day Monday)
- 03 Jun (03-07) - 1.43.0-wmf.8 – Dduvall + Ahmon (Brennen out)
- 10 Jun (10-14) - 1.43.0-wmf.9 – Brennen + Ahmon
- 17 Jun (17-21) – 1.43.0-wmf.10 – Jaime + Brennen
- 24 Jun (24-28) – 1.43.0-wmf.11 – Jeena + Jaime
- 01 Jul (02–04) – 1.43.0-wmf.12 – Antoine + Jeena
- 08 Jul (08-12) – 1.43.0-wmf.13 – Andre + Antoine (Ahmon + Dan out Friday)
- 15 Jul (15-19) – 1.43.0-wmf.14 – Ahmon + Andre (Dan Out)
- 22 Jul (22-26) – 1.43.0-wmf.15 – Dan + Ahmon (Ahmon out Thurs)
- 29 Jul (19-Aug 02) – 1.43.0-wmf.16 – Brennen + Dan (Ahmon out)
Team Discussions
[edit]Temp accounts is happening: https://phabricator.wikimedia.org/T368033
Thinking about adding k8s cluster into deployment-prep. Tell me why this is the worst thing ever!
- Helm charts may not apply
- should be another values.yaml
- Question: how does this relate to Code Stewardship request: https://phabricator.wikimedia.org/T215217 ?
- Group -1 is a sibling request https://phabricator.wikimedia.org/T369112
Production deployment servers to bullseye: https://phabricator.wikimedia.org/T364656 (not that active, I guess)
Fill your day with webforms
[edit]- Jul 17 (Today): Upward feedback in betterworks from you to tyler due <3
- Jul 24 (1 week): All-Hands 2025 head-count due: https://docs.google.com/forms/d/1Pdq9KKWBHdCja7hQ97aCqgtEO2b_GhI_y-3mijt_qTI/viewform?edit_requested=true
- Jul 30 (2 weeks-ish): Culture Amp "Culture of Review" 10 questions: https://wikimedia.cultureamp.com/adhoc/wikimedia?code=66844329fe50970033be6876
OKR talkie talk
[edit]- WE6.2
- WE6.2.1: Group -1
- Bryan is probably already talking to you :)
- Probably some overlap with WE6.2.5
- Giving QTEs what they need for testing via deployments
- Make it easier to put code into production faster
- WE6.2.2: Catalyst
- Need to close out Q4 and then sync with new users
- WE6.2.3: SpiderPig
- Threat analysis: https://docs.google.com/document/d/1HDFO_3EyK37Nb9yob8CN3wcwlI4GQJb5mycZloCjTg0/edit#heading=h.qy6gfvk2bh45
- Finalize security requirements, refactor scap backport, produce design doc
- Design: https://docs.google.com/document/d/1D1W-qNgFzKYyin5zgiWTR-8Zrc2_nqyTaQyIFVbm4OI/edit#heading=h.bd89949x67r2
- WE6.2.4: mw-on-k8s con't
- WE6.2.5: single-version images/routing
- WE6.2.1: Group -1
Let's do some inbox triage: https://phabricator.wikimedia.org/maniphest/query/7vRDrcVnt8OI/#R
🌻 Open source/Upstream contributions
[edit]