Jump to content

Wikimedia Release Engineering Team/Checkin archive/2024-08-14

From mediawiki.org


2024-08-14

[edit]

🏆 Wins/winterrogation

[edit]
https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Monthly_notable_accomplishments
August 2024
  • Fixed issue with deployment-deploy04 free space. Added a 40GB volume and copied /srv to it.
  • Buildkit 0.15.1 release deployed
  • Helped data-engineering Airflow DAGs with their Gitlab CI.
  • Rewrote remainder of make-container-image stuff in Python: https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/99 \o/
    • Scap invokes this repo during deployment
    • Current status: Create a php7.4 image + debugging packages
    • Future: php7.4 + php8.1
    • Single version images: there's a change in mw-config to override wikiversions.json
  • Updated train-dev to use debian:11 base image.
  • Kicked off nomination process to reboot the Toolforge standards committee (https://phabricator.wikimedia.org/T370474)
  • Moved a tool to toolforge build service
  • Fixed links in patchdemo for catalyst wikis
  • merged persistence for k8s patchdemo
  • Added read-only flag for patchdemo
  • Fixed a Phab code bug not checking user permissions creating a form
  • Merged more Phorge upstream stuff to get bugfixes + features once we pull, e.g. logging errors for broken Herald rules. See some stuff as deps: https://phabricator.wikimedia.org/T370266 (when downstream tasks exist)
  • Played with checking for active Phab accounts linked to locked WMF SUL accounts (TODO: other way round)
  • Started working on a Kubernetes cluster for deployment-prep using OpenTofu and Magnum as provisioning tools. Lots of things to figure out still, but a proof of concept cluster was provisioned, destroyed, and provisioned again. https://gitlab.wikimedia.org/bd808/deployment-prep-opentofu + deploymentpreps3
  • Scap deploy with rewrite of build-image script
  • Merged catalyst/patchdemo environment redirects


📅 Vacations/Important dates

[edit]
https://office.wikimedia.org/wiki/HR_Corner/Holiday_List#2024
https://wikitech.wikimedia.org/wiki/Deployments/Yearly_calendar
https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Time_off
  • Aug 02: Bryan
  • Aug 05-08: Dan
  • Fri 09 Aug – Global holiday: International Day of the World’s Indigenous Peoples
  • Aug 12: Dan
  • Mon 12 Aug-16: Ahmon out
  • Mon 12 Aug - Fri 23 Aug: Antoine
  • Aug 16: Bryan
  • Aug 23: Bryan
  • Aug 23: Jaime
  • Sat 24 Aug - 03 Sep: Brennen 🔥
  • Aug 30: Bryan
  • Sept 02: US Labor day (WMF US holiday)
  • Sept 06: Bryan
  • Sept 13: Bryan
  • Sept 18-19: Brennen Winfield
  • Sept 19-20, 23: Bryan Riot Fest in Chicago!
  • Sept23-27: Andre
  • Sept24: Dancy
  • Sept 27: Bryan
  • Sept 11-30 Jeena
  • Sept 27: Dan
  • Oct 03-06: WikiCon North America (Indianapolis)
  • Oct 6: Dancy
  • Oct 1-11 Jeena
  • Oct 14: Indigenous Peoples' Day (also Columbus Day) US Staff w/reqs

Future

[edit]

🔥🚂 Train

[edit]
https://versions.toolforge.org/
https://train-blockers.toolforge.org/
https://wikitech.wikimedia.org/wiki/Deployments/Yearly_calendar

Rotation

[edit]
  • 05 Aug (05-09) – 1.43.0-wmf.17 – Jaime + Brennen (Dan out, Global holiday Friday)
  • 12 Aug (12–16) – 1.43.0-wmf.18 – Jeena + Jaime (Ahmon out, Antoine out)
  • 19 Aug (19–23) – 1.43.0-wmf.19 – Andre + Jeena (Antoine out)
  • 26 Aug (26–30) – 1.43.0-wmf.20 – Antoine + Andre (Brennen out)
  • 02 Sep (02–06) – 1.43.0-wmf.21 – Ahmon + Antoine (US holiday Monday, Brennen out Tues)
  • 09 Sep (09–13) – 1.43.0-wmf.22 – Dan + Ahmon
  • 16 Sep (16–20) – 1.43.0-wmf.23 – Jaime + Dan (Brennen out)
  • 23 Sep (23–27) – 1.43.0-wmf.24 – Brennen + Jaime (andre out)

Team Discussions

[edit]

Brief demo of all the grafana data in digital ocean

[edit]
  • Background of this is trying to do horizontal pod autoscaling, but gitlab scaling has this
  • Kubernetes needed access to information collected in prometheus
  • Never done anything with grafana, gitlab job info may be in prometheus
  • How to get here:
 kubectl get ns
 kubectl --namespace monitoring get po
 kubectl --namespace monitoring describe po kube-prometheus-stack-grafana-cd65fd5f5-wtpfz 
 kubectl --namespace monitoring exec kube-prometheus-stack-grafana-cd65fd5f5-wtpfz -c grafana -- env
 kubectl --namespace monitoring port-forward service/kube-prometheus-stack-grafana 8081:80

Some things I looked at trying to get to it.

The config map: 

 kubectl --namespace monitoring describe configmaps kube-prometheus-stack-grafana

Seeeeccreetts (secrets): 

 kubectl --namespace monitoring get secrets kube-prometheus-stack-grafana
 kubectl --namespace monitoring get secrets kube-prometheus-stack-grafana -o jsonpath='{.data}'

logs: 

 kubectl --namespace monitoring logs kube-prometheus-stack-grafana-cd65fd5f5-wtpfz
 kubectl --namespace monitoring logs kube-prometheus-stack-grafana-cd65fd5f5-wtpfz -c grafana

I managed to find the secret in the env—it was not the secret from the secret: 

 kubectl --namespace monitoring exec kube-prometheus-stack-grafana-cd65fd5f5-wtpfz -c grafana -- env

Question:

Looming essential project roundup

[edit]
  • Zuul upgrade
  • Phabricator hosts on PHP 8.x
  • Phabricator on k8s???
  • Onboarding/offboarding
  • Deployment-prep beyond we6.2 (group -1/catalyst)
  • Attestation???
    • docker-pkg provenance
    • make-container attestation
    • Hooking this to deploy tools
  • helm deploy continuous deployment

🌻 Open source/Upstream contributions

[edit]
https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Upstream


Retrieved from "https://www.mediawiki.org/w/index.php?title=Wikimedia_Release_Engineering_Team/Checkin_archive/2024-08-14&oldid=6719024"