Wikimedia Release Engineering Team/Deployment pipeline/2019-06-06

2019-06-06

Last Time

• 2019-05-23
• Archive

General

• Helm 3
  • https://helm.sh/blog/helm-3-preview-pt1/
  • no more tiller
  • library charts

TODOs from last time

• stalled TODO various attack vectors document to start

• In progress TODO: support documention like the one tyler did for the portal and pipeline/helmfile and deployment
  • Martyav reached out on wiki https://wikitech.wikimedia.org/wiki/Talk:Deployment_pipeline
  • there may be a meeting?

• TODO docs for service docker container in beta

RelEng

• Dan is out for the next 6 weeks

• Pipeline .pipeline/config.yaml updates
  • Working for Blubber
    • Fancy: https://integration.wikimedia.org/ci/blue/organizations/jenkins/blubber-pipeline-rehearse/detail/service-pipeline-test-and-publish/14/pipeline/59
  • Some outstanding issues (lack of validation, etc.)
  • Probably not yet ready for wide use but potentially good enough for ORES? Not sure.
  • integration/config is not fully self serve but straightforward
    • https://gerrit.wikimedia.org/r/c/integration/config/+/510602

• Kask integration testing with Cassandra via the Deployment Pipeline
  • Sounds like we don't want to use the cassandra instance from the chart
  • Will need to provide ability to override values during helm install via pipeline
  • .pipeline/config.yaml pass additional chart values to helm, maybe

• Merging deployment-charts and local-charts
  • Move local-charts helm charts to a chart repository
  • deployment-charts meant to be local-dev first, originally
  • Does it still make sense to combine?
  • https://gerrit.wikimedia.org/r/plugins/gitiles/releng/local-charts/
  • Failing that, where should we put these charts?
  • Alex: Does it make sense to *not* have them combined?
  • Fsero: What is the scope of that repo?
  • Jeena: I would move only the charts from that repo into deployment-charts
  • Alex: Makes sense to publish those charts, will need a chart for MediaWiki soon
  • TODO jeena to make some patchsets to combine what makes sense

• CI Architecture and secure CI
  • What SRE needs are missing?
  • Link: https://docs.google.com/document/d/1vD3V4vrr2Jh_eFUBkHL3AnkzS3KmYZ1SxC3-MHUAf0o/edit
  • Alex: Next week, SRE summit, maybe make time for discussion

• Determine a standard way of installing MediaWiki lib/extension dependencies within containers
  • Please participate in discussion if you have opinions
  • Dan: I have an idea in there, want to find a middle-ground for a dependency resolver that is git-based and doesn't involve packagist

Serviceops

• sesstionstore deployed successfully
• looking into restrouter (probably early next quarter)
• wikifeeds (part of mobileapps/mcs) has been split off and requested to be deployed next FY
• moving on to termbox

Services

• Citoid/Zotero IP missing - https://phabricator.wikimedia.org/T225064
  • Need an addressable zotero to deploy with it
  • Create a service with a fixed IP, update the values.yaml with that fixed ip
  • Deploy as a helm chart dependency
  • Alex: subchart like with kask and cassandra, but that would still require some way of addressing zotero from citoid.
    • alternatively could deploy zotero in the same pod but that would pollute the citoid...
  • 'TODO .pipeline/helm.yaml delete, open task about above issue
• restrouter helm chart v1 - https://gerrit.wikimedia.org/r/#/c/operations/deployment-charts/+/512923/ 
  • Alex: needs a container image before we can review helm chart

—

As Always

• Release Pipeline Workboard
• Meeting notes