Wikimedia Release Engineering Team/SSD Sync Up/2019-08-06
Appearance
2019-08-06
[edit]Last Time: 2019-07-30 Goals: https://www.mediawiki.org/wiki/Wikimedia_Technology/Goals/2019-20_Q1#Release_Engineering
Deployment Pipeline
[edit]GOALS
[edit]- Migrate restrouter
- (Stretch): MobileContentService
- (Stretch): Preparatory MediaWiki config clean-up & static loading work
- Moving configuration to static configuration (e.g., json files rather than a bunch of PHP)
TODOs from Last Time
[edit]- Not done TODO Need to fix pipelinelib's base image -- make a task
- Not done TODO Add a changelog/NEWS file to blubber
- stalled brennen -- pipeline config validation
- now unlicked
Other Work
[edit]New CI
[edit]GOALS
[edit]- POCs of GitLab, Argo, and Zuul3 systems; evaluate options
- Done GitLab probably done
- ```TODO``` Lars: write up GitLab PoC evaluation
- Done Lars to set up meeting to discuss evaluation
- Not done
- Done GitLab probably done
- Dan working on argo
- parts of PoC working
- worked with upstream to implement more features, this time for git artifacts (both merged but not released)
- Moving with PoC running master
- Argo "workflow" k8s resource that defines task to run -- source that from the target repo as a PoC
- future work: transform .pipeline/config.yaml to workflow definition
- Argo Events to report back to gerrit...maybe
- Patched workflow controller instead of all components
- Lars: Not necessary to run upstream releases for PoC
- Dan working on argo
- Brennen working on zuulv3
- In progress
- (Still) Highly dependent on train
- James: is there an update about how easy it would be to migrate existing zuulv2 work to zuulv3?
- Brennen: my sense is a lot of work but doable (based on talking to Antoine) -- I have a feeling that it is a good deal less work than migrating to another system
- James: is the zuul migration like one we would end up building from scratch? Aka are we stacking the deck in favour of zuulv3 against argo and GitLab by doing this?
- Tyler: We're likely to build with nodepool using a static driver to map to the current jenkins agents; if we were doing it "properly" it'd be based on k8s/whatever.
- Dan: we should consider how difficult it is to migrate specific jobs in the evaluation
- Brennen: Worth trying k8s with zuul / nodepool?
- Lars: Pipelinelib written in groovy cannot be used outside of jenkins, so we should consider how difficult this will be to migrate
- James: We have 423 JJB jobs + 8500+ of lines of layout for ~1400 specific repos -- although they're maybe not "good" -- migration is a chance to fix things
- Dan: if pipelinelib were the new way to do things, we should evaluate how easy that is to implement in the different systems
- Brennen working on zuulv3
- Document an implementable architecture for what we want in new CI
- Lars: Will have new version hopefully for Thursday (didn't, still working on this)
TODOs from Last Time
[edit]Other Work
[edit]Local Development
[edit]GOALS
[edit]- Migrate local-charts to deployment-charts
- Instantiate testing and linting of helm charts
- Preliminary work on a CLI for setup/management
- Progressing slowly: https://gerrit.wikimedia.org/r/c/releng/local-charts/+/525563
TODOs from Last Time
[edit]- mediawiki/core blubber
- Done TODO add branch filter for this job to only trigger on master
- Ready for review? https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/525972/
- Blubber doesn't allow modification of files
- runs: {insecurely: true}
Other Work
[edit]- Porting from local-charts to deployment-charts
- In progress MediaWiki
- Done Move mediawiki chart from local-charts to deployment-charts repo
- https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/522584
- Wasn't this blocked on? (no, we will update the chart after that work is done)
- https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/522584
- Update local-charts repository to use mediawiki chart from deployment-charts repo Template:Has patchset
- Mukunda left feedback
- Done Move mediawiki chart from local-charts to deployment-charts repo
- In progress MediaWiki
- In progress Restbase
- Move restbase chart from local-charts to deployment-charts repository
- Update local-charts repository to use restbase chart from deployment-charts repo
- In progress Restbase
- Local Dev CLI
- Mukunda has updated patchset
- Refactoring -- expect patchset update later today