Wikimedia Release Engineering Team/Deployment pipeline/2020-09-02
2020-09-02
[edit]Always
[edit]- [1]
- Archive
- Task tree: https://phabricator.wikimedia.org/T198901
TODOs from last time
[edit]- Meeting with Fandom happened yesterday...
- TODO Tyler to ask Fandom if they have a recording of the meeting
Done they do :)
General
[edit]- Moritz mentioned that restbase is using old mathoid
- What does this have to do with restbase?
- Restbase is a reverse proxy in front of mathoid
- There is an issue with mathoid: https://phabricator.wikimedia.org/T207535
- Holger might be able to deploy services
- https://wikitech.wikimedia.org/wiki/Deployments_on_kubernetes#Deploying_with_helmfile
RelEng
[edit]- Jeena has a few automation patchsets in progress:
- Magic image version update https://gerrit.wikimedia.org/r/619833
- https://gerrit.wikimedia.org/r/c/integration/pipelinelib/+/622231
- goal: have this happen as part of the pipeline
- Dan has been thinking about container composition and workflows.
- See https://phabricator.wikimedia.org/T259817#6395133 for current thinking.
- Creating from scratch images and https://phabricator.wikimedia.org/T255595
- Start integrating for MediaWiki for production early
- Satisfies the need for a build step
- Current status: l10n cache rebuilding updates. Currently depends on a full MediaWiki installation. Did a deep dive on rebuildLocalisationCache.php with xdebug.
- Clears resourceloader cache as part of rebuild, but is likely incorrect since we update Wikiversions *after* l10n update
- Scap purges cache for resourceloader as well post update -- that seems correct
- removing resourceloader purge from rebuildLocalisationCache removes the need for a database connection
- the next step is localisation hooks, but no production extensions use those hooks
- deprecating those hooks, removing the resourceloader clearing, makes the l10nupdate much faster (hopefully), but it's a deep rabbithole
- Feedback for image building
- Scratch images in registries to integrate images later on
- Makes use of local caches and blubber
- Would love additional review
- Alex will send to team
- Blubber patch to support scratch images
- Investigating MW LocalisationCache things. So far:
- Process is database independent if we remove the ResourceLoader cache clear (doesn't seem like a good thing to do in that process anyhow)
- No production-deployed extensions actually implement the two hooks defined by LC rebuild
- These things mean it may be more possible to factor out LC rebuild from MW into a separate/faseter (maybe non-MW-PHP specific) process and compile LC files earlier in an MW image building pipeline
- Need to dig deeper.
Serviceops
[edit]- SRE will be reinitializing the eqiad kubernetes cluster next week since all of eqiad is depooled
- Working on a policy document for k8s and calico version upgrades
- focused on k8s upgrades, Docker upgrades later
- Work on push-notifications resumed.
- Resuming work on apertium moving to the pipeline
- this and graphoid the final scb services