Data Platform Engineering/Data Platform SRE/Process

It's not Scrum, it's not Kanban, but we do have a process and this page describes it. This is meant to be descriptive, not prescriptive. Our process evolves all the time as we improve it, documentation is likely to be lagging behind.

Principles

• As an SRE team, we are interrupt driven. We have projects and priorities, but the reality of operations takes precedence.
• We want to share knowledge and responsibilities. More than one person should be involved in any project. Team members who don't know anything about a specific task are encouraged to take it on an learn. This makes us slower in the short term, but faster in the long run.
• Engineering manager is there to bring clarity on the work that needs to be done, and on the current priorities. Engineering manager isn't there to assign work to specific people. Everyone takes ownership of our platform and of our work.

Tracking work

Our high level priorities are documented on wiki. Our long term backlog is in Phabricator on the Data-Platform-SRE board. Our current work is organized in 3 weeks iterations (not sprints). There is no stable link to our current iteration, but there is a link to the current iteration from the menu of our backlog phab board.

Incoming work

The "Incoming" column of our backlog board is reviewed multiple times per week. Tasks are either pushed directly to our current iteration, or prioritized and moved to the appropriate column on the backlog board. In case of urgent work (incident, unblocking someone, ...), this process is not reactive enough, you should ping us directly on the # data-platform-sre Slack channel, or on #wikimedia-data-platform ^connect channel on IRC.

Regular meetings

• DPE SRE - triage / planning
  • weekly
  • sharing of information
  • alignment on priorities
  • review ongoing work in a standup inspired fashion
  • review phab board depending on time constraints
• Retrospective
  • every 2 weeks (but is sometimes cancelled due to conflicting meetings)
• Learning circles
  • monthly (but cancelled if no topic is selected)
  • sharing of knowledge within the DPE SRE team