Wikimedia Release Engineering Team/CI Futures WG/Meetings/2019-05-01
A sync up with TechComm.
This meeing is meant to be a quick check-in, but people are missing
- need to replace current infrastructure tooling
- Zuul being the main scheduler, currently running a forked deprecated version (v2) but its now on v3 and works very differently
- weâve locally patched our deprecated version
- Three options being evaluated, slowly this quarter, more intently next quarter
- CI Futures WG Report
- goal is to have CI jobs like Blubber in repos
- Marko: curious that Gitlab is being considered, as a non-installed but rather tried online option
- one of the hard criteria is that it be installed
- Guiseppe:has a code review part, which is most important part
- Lars: installed version is slightly different from their SaaS product, but is close enough
- Greg: no appetite for changning code review process right this moment
- _____
- Joe: plan to use it for single projects? pipeline and other projects to be sure it works? (yes)
- have you thought of having different levels of access for different users?
- do we need a separate jenkins instance just for putting somethings into production?
- Lars: we haven't explicitly looked into this but seems a good requirement
- Timo: sounds reasonable. post merge jobs are always a security problem. it's currently stable though
- we finally found something that is up to our high standards, but not a given in other systems
- managing credentials, access post-merge, documentation pipeline-- unsure if we can do this for pipeline
- but maybe we can for zuul and jenkins
- Greg: zuul v3 no longer uses jenkins
- Marko: did not find hhow you plan on testing this - for scalablity. jenkins has a big backlog of work
- Joe: it's most of the time zuul that creates issues
- greg: scalablity is hard to test...can't test scale without having scale
- Timo: Happy to see Jenkins be dropped (heard for a very long time indeed) - in that case I wonder where credentials are stored with Zuul.
- Credentials and storing of logs are basically the two things we use Jenkins for at this point.
- The rest is all Zuul/Gearman/Docker, we use almost none of Jenkins.
- Timo: if we use all three, what credentialing do we need?
- executuing a shell script for credentialing (we use jenkins for this now)
- Daniel: from a user perspective, I have a few things that might be edge case that shouldn't be forgotten
- how tests for vendor works? patches against core and patches against vendor? in theory they break each other...this patch would need to support this
- patches get pulled into CI and some other extension is running the code? extension not libraries?
- Greg: need to also take into account:
- credential store
- depends-on feature
- vendor
- build step stuff from Readers Web
- Roan: zuul also manages multile patches, tests for dependencies. we'd need to do that as well
- Timo: dependant pipeline work.
- pre-merge testing bit (just testing the patch for dependencies) - zuul cloner does this (non-trival)
- on the merge itself - the depenant pipeline. even if plus 2 a patch, zuul will assume that both patches will pass.
- avoiding the master branch from getting in a state where tests don't pass
- People say Zuul is a constant source of issues, part of that is because dependency is complicated in our CI system
- Greg: many debates on the pipeline.. :)
- it's a nice to have, but not a requirement, we should be able to mitigate it
- ACTION Items:
- CI WG: review the open questions (credential store, depends-on feature, vendor, build step), come back to techcomm with some responses