2017-02-21

Antoine, Jean-René, Giuseppe, Marko, Tyler

Background Notes

Problems
- Releng October 2016 offsite
- CI pipeline != production != development
- CI/Beta situation -- we want to make them closer to production, want developers to be able to run something close locally
- Magic containers fix everything magically — also, lots of work: https://www.mediawiki.org/wiki/File:Containerized_continuous_delivery_2017_concept.png

Notes from Giuseppe:

Need a single cli tool that generates a standard, WMF-specialized dockerfile for at least: php/hhvm, python, node, java
Existing debian packages of libraries we already use should be preferred by the tool whenever possible
Only use images coming from https://registry.wikimedia.org/ (official docker registry, public already)
- Images build on and published from copper (machine ops use to build packages/images etc)
NO private data in container images, ever. They are all going to be public by default
Integration layers need a *lot* of thought
Dockerfile generation can be tricky
Images will not be uploaded by users, they just upload the repo, the official artifact will be built by CI or "releases"
- How to mark releases?
- gerrit-specific?
CI pipeline from pearson https://github.com/pearsontechnology/deployment-pipeline-jenkins-plugin

Base images to be maintained by ops

includes limited APT sources, etc.
what happens when a new base image is built/registered?
- rebuild all dependent images and upload to registry (triggered/automated in CI)
do we need a private registry? seems there's some facility for this already

Some kind of unified/standardized "build manifest" that abstracts away the problematic Dockerfile

Example given https://github.com/pearsontechnology/deployment-pipeline-jenkins-plugin#buildbitesize
Exposes limited instructions for installing dependencies
k8s does rolling deploy

Tests with private data (eg MediaWiki security patches)? Part of Docker registry can be made private.