Wikimedia Release Engineering Team/Deployment pipeline/2019-05-09

Last Time

• 2019-04-11
• Archive

General

—

TODOs from last time

• TODO what are our annual plans WRT to this project

• Template:Stalled TODO various attack vectors document to start

• In progress TODO: support documention like the one tyler did for the portal and pipeline/helmfile and deployment
  • https://wikitech.wikimedia.org/wiki/Deployment_pipeline now exists, https://wikitech.wikimedia.org/wiki/Continuous_Delivery has been deleted.
  • reached out to technical writer, in very preliminary stages for next quarter
  • think about other documentation needs beyone Deployment_pipeline main page
  • Anything we can do here without stepping on toes?
    • preliminary research happening
    • work with srodlund about what the portal is
    • starting on the real work of the documentation next week

• TODO docs for service docker container in beta

•  Done TODO: announce email
  • https://lists.wikimedia.org/pipermail/wikitech-l/2019-April/091976.html
  • Feedback?
    • Nothing major, mostly queries about MediaWiki containerisation.
  • Some talk about Migrate Beta cluster services to use Kubernetes

RelEng

• pipelinelib review meeting after this -- hopefully make progress towards .pipeline/config.yaml
  • 502918 - pipeline: Builder and stage implementation
  • 502919 - pipeline: Provide a rickety but useful system test
  • 502917 - pipeline: Directed graph execution model
• Docs are published for pipelinelib
  • https://doc.wikimedia.org/pipelinelib/
• deployment to staging -- we deploy to ci namespace on neon, but we also run helm delete
  • to deploy to staging in the general case: deploy to staging with a namespace that is the same name as the service
  • problems and edgecases with this are repos with multiple services that need to be deployed; e.g., eventgate-ci
  • pipelinelib could potentially allow for multiple services to be deployed from a single repository; i.e., .pipeline/config.yaml
  • for helm test, consider using:
    • https://github.com/kubernetes-sigs/kind

• Local development images using blubber:
  • https://gerrit.wikimedia.org/r/c/mediawiki/services/parsoid/+/508439
  • https://gerrit.wikimedia.org/r/c/mediawiki/core/+/508392

Serviceops

• kask got swagger/openapi spec. Merge pending
  • how do we do this for staging since kask needs cassandra?
  • setup vms? we have hardware possibly...
  • citoid has similar problems, but if we could pass a proxy to the helm test
    • Dan: we could provide configuration to pass to helm install or helm test in the pipeline
  • something to think about: things we have no plan to move to k8s (e.g., mysql) how do we deal with those in staging -- resource problems in general
    • maybe possible to work around the helm test part with some hack
    • e.g., restbase works around this by heavily mocking responses
    • this may be harder with MediaWiki
    • quibble currently works around this
    • MediaWiki has this problem in the testrunner a bit i.e., unit tests fail if it tries to write to the database -- other things write to sqlite
  • stress testing for projects
    • staging, in theory, in the future should be able to be used for some portion of stress testing
    • two problems: (1) providing servers for data (2) providing the data -- the 2nd one will be difficult
  • "staging"
    • is this a persistent k8s pre-production environment
    • is this a environment where we could mock important services
    • these two things serve two different testing strategies

• some work on making kask more easily working on minikube, courtesy of incubator/cassandra chart
• Template:Started Tech talk scheduled from July 10
• new registryha live, no complains so far
  • https://phabricator.wikimedia.org/T209271

Services

—

As Always

• Release Pipeline Workboard
• Meeting notes