Jump to content

Wikimedia Release Engineering Team/SSD Sync Up/2019-08-06

From mediawiki.org

2019-08-06

[edit]

Last Time: 2019-07-30 Goals: https://www.mediawiki.org/wiki/Wikimedia_Technology/Goals/2019-20_Q1#Release_Engineering

Deployment Pipeline

[edit]

Workboard

GOALS

[edit]
  • Migrate restrouter
  • (Stretch): MobileContentService
  • (Stretch): Preparatory MediaWiki config clean-up & static loading work 
    • Moving configuration to static configuration (e.g., json files rather than a bunch of PHP)

TODOs from Last Time

[edit]
  • N Not done TODO Need to fix pipelinelib's base image -- make a task
  • N Not done TODO Add a changelog/NEWS file to blubber

Other Work

[edit]

New CI

[edit]

GOALS

[edit]
  • POCs of GitLab, Argo, and Zuul3 systems; evaluate options
    • Yes Done GitLab probably done
      • ```TODO``` Lars: write up GitLab PoC evaluation
      • Yes Done Lars to set up meeting to discuss evaluation
          • Not done
    • Dan working on argo
      • parts of PoC working
      • worked with upstream to implement more features, this time for git artifacts (both merged but not released)
      • Moving with PoC running master
      • Argo "workflow" k8s resource that defines task to run -- source that from the target repo as a PoC
      • future work: transform .pipeline/config.yaml to workflow definition
      • Argo Events to report back to gerrit...maybe
      • Patched workflow controller instead of all components
      • Lars: Not necessary to run upstream releases for PoC
    • Brennen working on zuulv3
      • In progress In progress
      • (Still) Highly dependent on train
      • James: is there an update about how easy it would be to migrate existing zuulv2 work to zuulv3?
      • Brennen: my sense is a lot of work but doable (based on talking to Antoine) -- I have a feeling that it is a good deal less work than migrating to another system
      • James: is the zuul migration like one we would end up building from scratch? Aka are we stacking the deck in favour of zuulv3 against argo and GitLab by doing this?
      • Tyler: We're likely to build with nodepool using a static driver to map to the current jenkins agents; if we were doing it "properly" it'd be based on k8s/whatever.
      • Dan: we should consider how difficult it is to migrate specific jobs in the evaluation
      • Brennen: Worth trying k8s with zuul / nodepool?
      • Lars: Pipelinelib written in groovy cannot be used outside of jenkins, so we should consider how difficult this will be to migrate
      • James: We have 423 JJB jobs + 8500+ of lines of layout for ~1400 specific repos -- although they're maybe not "good" -- migration is a chance to fix things
      • Dan: if pipelinelib were the new way to do things, we should evaluate how easy that is to implement in the different systems
  • Document an implementable architecture for what we want in new CI
    • Lars: Will have new version hopefully for Thursday (didn't, still working on this)

TODOs from Last Time

[edit]

Other Work

[edit]

Local Development

[edit]

GOALS

[edit]

TODOs from Last Time

[edit]

Other Work

[edit]
    • In progress In progress Restbase
  • Local Dev CLI
    • Mukunda has updated patchset
    • Refactoring -- expect patchset update later today