Jump to content

Wikimedia Release Engineering Team/Checkin archive/20190603

From mediawiki.org


2019-06-03

[edit]

Vacations/Important dates

[edit]
https://office.wikimedia.org/wiki/HR_Corner/Holiday_List
How to do it
  • June 6-7 - Brennen, Apogaea
  • June 10th - Antoine, Pentecost -- see https://en.wikipedia.org/wiki/Eastertide for Antoine/France Easter holidays
  • June 10 – July 21 - Dan leave (6 weeks, then additional leave later)
  • June 19 (Juneteenth) - US Staff - on a Wednesday!?
  • June 20 - Željko, Corpus Christi
  • June 25 - Željko, Statehood Day
  • July 4 (US Independence Day) - US Staff
  • July 22 - August 9 - Željko vacation
  • July 22 - Lars, Midsummer
  • August 7–19 - James off (inc. Wikimania)
  • August 12 - September 8 - Dan leave
  • August 12 (Glorious Twelfth) - US Staff
  • August ??? - ??? - Antoine
  • August 14–18 - Wikimania
    • Attending: James, Lars, ? …
  • August 15 - Željko, Assumption of Mary
  • August 25 - September 4 - Brennen vacation
  • September 2 (Labor Day) - US Staff
  • October 14 (Indigenous Peoples' Day) - US Staff
  • November 11 (Veterans' Day) - US Staff
  • November 28–29 (Thanksgiving) - US Staff
  • December 6 - Lars, Finnish Independence Day
  • December 25–31 (Christmas) - US Staff
  • December 25-26 - Lars, Christmas
  • 2020 January 1 (New Year's Day) - US Staff, Lars

Rotating positions

[edit]

Train

[edit]
Maniphest query for deployment blocker tasks: https://phabricator.wikimedia.org/maniphest/query/s3KW8bpsXhYF/#R
  • May 27 - wmf.7 - Zeljko 😢
  • June 03 - wmf.8 - Zeljko 😭
  • June 10 - wmf.9 - No Train (SRE Summit)
  • June 17 - wmf.10 - Mukunda (but Juneteenth on the Wednesday? Yes.)
  • June 24 - wmf.11 - Mukunda
  • July 1 - wmf.12 - No train (Fourth of July)
  • July 8 - wmf.13 - Tyler
  • July 15 - wmf.14 - Tyler
  • July 22 - wmf.15 - Antoine
  • July 29 - wmf.16 - Antoine
  • Aug 5 - wmf.17 - one of Mukunda/Tyler (Antoine and Zeljko on vacation)
  • Aug 12 - wmf.18 - No Train (Wikimania) 😳 Last year we discussed not having train during Wikimania https://wikitech.wikimedia.org/wiki/Incident_documentation/20180717-Train
  • Aug 19 - wmf.19 - Zeljko (after Wikimania) 😱
  • Aug 26 - wmf.20 - Zeljko

SoS

[edit]
  • Zeljko 4eva! :)


Team Business

[edit]

Timespent spreadsheet

[edit]
For the avoidance of doubt: fill out the sheet week number for the previous week


Book club

[edit]


Spring Offsite

[edit]

Follow-ups:

  • Greg: email mark about capex request for next year for pipeline
    • I'm actually not sure what this is about/what the ask is, help?!
    • "staging" pipeline?
    • Production access?
    • CapEx budget now locked.
  • Mukunda: talk with Timo and Fillipo about our prioritized of feature requests for LMM
  • Greg: announce that RelEng is backup only for SWAT (removal of person’s names from getting pinged everytime on IRC) and we’ll start working on automating the train
    • Still need to do Q4 goals...table this “doing” until Q1?
    • Greg will send a signed email if someone writes it up ;)
      • Željko will write the e-mail this week - done
      • Greg to follow-up...


Fall Offsite + TechConf19

[edit]


Monthly reflection on accomplishments - May '19 edition

[edit]
https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Monthly_notable_accomplishments
Add as you have them!

Annual Planning

[edit]
  • ...


Annual Reviews

[edit]

Overview: https://office.wikimedia.org/wiki/FY_2018-19_Annual_Review_and_Retrospective


Deadlines

[edit]

Everyone:

  • Starting now: You and I discuss who your peer reviewers should be
  • April 26th: Enter your peer reviewers into Namely (please run them by me first)
  • May 17th: Deadline to complete self-reviews, peer reviews, and reviews of your manager.
  • May 20th: I start reviewing the peer reviews and writing my feedback on you.

Non SafeGuard (aka US Employees):

  • June 14th: Deadline for managers to complete all 1:1 meetings with direct reports and provide written feedback in Namely.

SafeGuard:

  • June 14th - Managers of those employed by Safeguard submit their reviews to HR for submission to Safeguard
  • July 12th - Deadline to have a 1:1 and share final manager review with direct report in Namely


Incoming/Needs attention

[edit]
  • REL1_33 branching for extensions: https://phabricator.wikimedia.org/T220653
    • Reedy said he'll move forward with rc0 announcement soon.
    • Mukunda tried to run ther script but it ran into trouble. Will re-try, manually.
    • Switching on HTTP Auth again still seems blocked. Barricade should help with this; review when Tyler gets back.
      • Update 2019-06-03: Fighting fires last; should be able to do this week.
  • CI Node 10 migration – let's JFDI? https://phabricator.wikimedia.org/T222406 Will need to pair with a CI expert (hashar?)
    • James and Antoine to pair next week.
      • Update 2019-06-03: In progress. Paired on it for a couple of hours so far today, more to come.

Scrum of Scrums

[edit]

Incoming from last week

[edit]
  • Blocking:


Outgoing this week (wrong section heading level is on purpose for copy/pasting into Scrum of Scrums etherpad

[edit]

Release Engineering

[edit]

Callouts

[edit]
  • Release Engineering


Train status and happenings

[edit]
https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Roles#Train_Conductor
  • Need to fix scap clean :\
    • thcipriani has a crappy fix in mind until http tokens in gerrit are back
      • Any idea when HTTP tokens will come back? Weeks? Months? Never? :-(
      • ~Weeks
    • 2019-05-06: cleaned up stuff last week on deploy hosts, just not the gerrit branches
    • 2019-05-13: …
    • 2019-06-03: upstream issues/patches we want resolved before doing this
  • 1.33 branch cut for extensions is blocked (except tarball ones, which James did manually)
    • 2019-05-06: Mukunda to do it this week
    • Greg: email Cindy re process of this release
    • 2019-05-13: We talked on Thursday. Mukunda will review hexmode's work, Cindy will email Greg with plan of action re timeline.
    • 2019-06-03: See above.


Quarterly Goals for Q4

[edit]

https://www.mediawiki.org/wiki/Wikimedia_Technology/Goals/2018-19_Q4

TEC1 (Maint): Outcome 1 / Output 1.1

[edit]
GOAL: Undeploy the CodeReview extension.
WHO: James, need help from CPT
  • James will ping CPT about this this week (April 8th)
    • … and again w/c 15 April.
    • … and again w/c 6 May (in SoS).
    • … and again w/c 27 May (in SoS).


TEC1 (Maint): Outcome 1 / Output 1.1

[edit]
GOAL: Setup 1-3 of the CI WG options (Zuul v3, Argo, GitLab)
WHO:
  • Focus on a couple noteworthy repos: e.g.,
    • core
    • extensions
    • ops/puppet
  • Maybe setup in serial, i.e., a week per evaluation
  • Questions:
    • RelEng/Extended working group?
      • At least in the WG eval it was good to have non-familiar people
      • But maybe with the setup of options it might be beneficial to have experienced with current setup people.
      • Folks outside the original working group to join-in to setup options; people TBD
    • Do we need a rubric before we do this prototyping? (yes)
      • DONE lars to work on rubric week of 2019-04-01
        • See email 2019-04-08
    • CI arch doc in team google drive now, open for feedback
  • 2019-05-06: Feedback from Android. Working on an arch document. Do in Q1?


TEC3 (Pipeline): Outcome 1 / Output 1.2

[edit]
GOAL: Instrument Quibble for data collection
WHO: Mukunda, Antoine


TEC3 (Pipeline): Outcome 1 / Output 1.2

[edit]
GOAL: Create a graph where time is spent and make a prioritized list for improvements.
WHO: Mukunda, Antoine
  • Blocked

TEC3 (Pipeline): Outcome 1 / Output 1.2

[edit]
GOAL: Prepare the Deployment Pipeline for changes to our CI tooling.
WHO: ???, ???
  • Blocked by not having new CI tooling yet


TEC3 (Pipeline): Outcome 3 / Output 3.1

[edit]
GOAL: Create a .pipeline/config.yaml standard to give users more control over how their tests are run in the pipeline and allow the easy saving of artifacts at pipeline completion. (RelEng)
WHO: Dan, Tyler, ???
  • Implementation is working, but in testing a Blubber .pipeline/config.yaml there are some glaring deficiencies
  • Long term concerns about Groovy implementation include:
    • Dependencies on Jenkins and many plugins
    • Groovy CPS is a huge pain to debug and it's rarely clear that CPS is the issue when things go awry; Instead, the code just executes in unexpected ways.


TEC3 (Pipeline): Outcome 3 / Output 3.1

[edit]
GOALS:
Adopt more services into Deployment pipeline - task T212801
Wikidata Termbox SSR, Kask for Session Storage Service, cpjobqueue (stretch), ORES (stretch)
WHO: Dan, Tyler, Lars

There are tasks: https://phabricator.wikimedia.org/T220403

  • changeprop


  • Wikidata Termbox SSR
  • Kask for Session Storage Service
  • cpjobqueue (stretch)

TEC12 (DevProd): Outcome 1 / Output 1.1

[edit]
GOAL: Provide an "Official" Docker base image for local development of MediaWiki based on the production tooling.
WHO: Jeena, Brennen
https://phabricator.wikimedia.org/T212449
  • Done for MediaWiki, for some values of "done" and "MediaWiki". Production-likeness needs considerable work.

TEC13 (Code Health): Outcome 1 / Output 3

[edit]
GOALs: Presentation/session(s) at the Wikimedia Hackathon on the current state of Code Health projects (technical debt and code stewardship)
WHO: JR


TEC13 (Code Health): Outcome 1 / Output 1.1

[edit]
GOAL:
Publish a re-imagination of the Review Queue process.
Develop and implement metrics around task and code-review responsiveness
WHO: Greg, JR (and Andre)
  • Closed call for participation in Code Review workgroup. Working on scheduling kickoff meeting.


= TEC13 (Code Health): Outcome 4 / Output 4.2

[edit]
GOALs:
Expand SonarQube reporting into CI infrastructure
Perform SonarQube analysis on all extensions
Engage user communities in direct feedback solicitation
WHO: JR, Zeljko, Code Health Metrics
  • We currently have 6 extensions in the new pipeline, will be at 10 within the week. Planning to push for the rest of extensions starting next week.
  • Core is not currently in a state to be added due to extended unit test run times. Project (non CHMWG) underway to seperate unit for integration tests.
  • Looking to expand to other areas of analysis such as Python/SCAP. This will require some wramp up and assistance from others.


Other non-goal work

[edit]

Release MW 1.33

[edit]

Selenium

[edit]
  • T223774 The first Selenium test for WikibaseCirrusSearch - started at the hackathon, have to finish it

Gerrit

[edit]

Phabricator

[edit]

Jenkins

[edit]

QA/Code Health

[edit]
  • Community project started for Core to start splitting unit and integration tests.

SCAP

[edit]

Standup!

[edit]

Antoine

[edit]


Brennen

[edit]
  • What I plan to do this week
  • Remove Docker dependency from local-charts: https://phabricator.wikimedia.org/T223715
  • Get various incoming local-dev patchsets merged.
  • What I'm blocked on
    • Nada
  • Other?
    • Off-grid June 6-9
    • If I seem to be operating at like 20% of my cognitive baseline, it's 'cause allergy season just started in Colorado.


Dan

[edit]
  • What I plan to do this week
    • Wrap up before leave
      • How to best pass off pipelinelib work?
      • Meeting with Analytics delayed. They want a document of use cases. JR or Mukunda may want to pick this up or wait. Either works for me
        • JR: "We'll give it a go"
    • Get calendar in order and decline meetings
  • What I'm blocked on
    • Nothing really
  • Other?
    • Thoughts on MediaWiki dependency resolution/installation
      • https://phabricator.wikimedia.org/T193824#5227012
      • A "dev-requires" field has been added to extension.json schema, and a maintenance/checkDependencies.php script is now in core
      • I rambled about an idea of aggregating dependencies into a central service on postmerge, a service that could potentially map extension/skin version contraints to gitremotes/refs. This needs fleshing out if it's generally sane.


Greg

[edit]
  • What I plan to do this week
    • Annual Reviews writing
    • TechConf19
    • DMV appointment on Thursday :-(
    • Read book, hopefully!
  • What I'm blocked on
    • The linearity of time.
  • Other?
    • SRE summit week of Jun 9


James

[edit]
  • What I plan to do this week
    • Node 10 CI stuff
    • Pipeline documentation
    • Unit vs. Integration test split help
  • What I'm blocked on
    • Extension undeployment stuff, as before.
  • Other?
    • Whatever blows up.


Jean-Rene

[edit]
  • What I plan to do this week
    • More time on Code Stewardship review
    • Continued Code Review workgroup setup/planning
    • Work with CPT on Integration Testing framework decision
  • What I'm blocked on
  • Other?


Jeena

[edit]
  • What I plan to do this week
    • Organize local-charts backlog/workboard
    • Create phabricator task for liveness/readiness
    • Discuss/plan interface for interacting with local-charts
    • Read book
  • What I'm blocked on
  • Other?


Lars

[edit]
  • What I plan to do this week
    • reading CD book
    • update CI arch doc, and reach out more for more feedback
    • look at installing GtLab somewhere
  • What I'm blocked on
  • Other?


Mukunda

[edit]
  • What I'm blocked on
    • yaks
  • Other?


Tyler

[edit]
  • What I plan to do this week
    • get things from dan's brain
    • l10n checker patches
    • scap release
    • reply on "rethinking deployment" task
    • dig out from my vacation email backlog
    • dcausse annual review
  • What I'm blocked on
  • Other?


Zeljko

[edit]
  • What I plan to do this week
    • T220733 1.34.0-wmf.8 deployment blockers
    • T223774 The first Selenium test for WikibaseCirrusSearch
  • What I'm blocked on
  • Other?
    • Didn't do a swat since the hackathon :D


Grooming

[edit]

Team Kanban Board Review and Triage

[edit]


Once / month-ish review of backlog(s)

[edit]


Kanban stats

[edit]
Burnup chart
Retrieved from "https://www.mediawiki.org/w/index.php?title=Wikimedia_Release_Engineering_Team/Checkin_archive/20190603&oldid=4351843"