Jump to content

Wikimedia Release Engineering Team/CI Futures WG/Meetings/2019-05-01

From mediawiki.org

A sync up with TechComm.

This meeing is meant to be a quick check-in, but people are missing

  • need to replace current infrastructure tooling
  • Zuul being the main scheduler, currently running a forked deprecated version (v2) but its now on v3 and works very differently
    • we’ve locally patched our deprecated version
  • Three options being evaluated, slowly this quarter, more intently next quarter
    • CI Futures WG Report
    • goal is to have CI jobs like Blubber in repos
  • Marko: curious that Gitlab is being considered, as a non-installed but rather tried online option
    • one of the hard criteria is that it be installed
    • Guiseppe:has a code review part, which is most important part
    • Lars: installed version is slightly different from their SaaS product, but is close enough
  • Greg: no appetite for changning code review process right this moment
    • _____
  • Joe: plan to use it for single projects? pipeline and other projects to be sure it works? (yes)
    • have you thought of having different levels of access for different users?
    • do we need a separate jenkins instance just for putting somethings into production?
  • Lars: we haven't explicitly looked into this but seems a good requirement
  • Timo: sounds reasonable. post merge jobs are always a security problem. it's currently stable though
    • we finally found something that is up to our high standards, but not a given in other systems
    • managing credentials, access post-merge, documentation pipeline-- unsure if we can do this for pipeline
    • but maybe we can for zuul and jenkins
  • Greg: zuul v3 no longer uses jenkins
  • Marko: did not find hhow you plan on testing this - for scalablity. jenkins has a big backlog of work
  • Joe: it's most of the time zuul that creates issues
  • greg: scalablity is hard to test...can't test scale without having scale
  • Timo: Happy to see Jenkins be dropped (heard for a very long time indeed) - in that case I wonder where credentials are stored with Zuul.
    • Credentials and storing of logs are basically the two things we use Jenkins for at this point.
    • The rest is all Zuul/Gearman/Docker, we use almost none of Jenkins.
  • Timo: if we use all three, what credentialing do we need?
    • executuing a shell script for credentialing (we use jenkins for this now)
  • Daniel: from a user perspective, I have a few things that might be edge case that shouldn't be forgotten
    • how tests for vendor works? patches against core and patches against vendor? in theory they break each other...this patch would need to support this
    • patches get pulled into CI and some other extension is running the code? extension not libraries?
  • Greg: need to also take into account:
    • credential store
    • depends-on feature
    • vendor
    • build step stuff from Readers Web
  • Roan: zuul also manages multile patches, tests for dependencies. we'd need to do that as well
    • Timo: dependant pipeline work.
    • pre-merge testing bit (just testing the patch for dependencies) - zuul cloner does this (non-trival)
    • on the merge itself - the depenant pipeline. even if plus 2 a patch, zuul will assume that both patches will pass.
    • avoiding the master branch from getting in a state where tests don't pass
    • People say Zuul is a constant source of issues, part of that is because dependency is complicated in our CI system
  • Greg: many debates on the pipeline.. :)
    • it's a nice to have, but not a requirement, we should be able to mitigate it
  • ACTION Items:
    • CI WG: review the open questions (credential store, depends-on feature, vendor, build step), come back to techcomm with some responses
Retrieved from "https://www.mediawiki.org/w/index.php?title=Wikimedia_Release_Engineering_Team/CI_Futures_WG/Meetings/2019-05-01&oldid=3612075"