Program Goals and Status for FY18/19

TEC13: Code Health

Goal Owner: Greg Grossmeier
Program Goals for FY18/19: The program shares its goals/vision with the Code Health Group, which is: work on efforts that universally improve the lives of engineers and their ability to write products with shorter iteration time, decreased development effort, greater stability, and improved performance. Working closely with the Code Health Group, the Code Health Program will focus on the software engineering processes and practices that enable both experienced and junior technical contributors to deliver high value/quality features to the various Wikimedia projects in the most productive way possible
Annual Plan: TEC13: Code Health
- Primary Goal is Knowledge as a Service: Evolve our systems and structures
- Tech Goal: Sustaining

Q1 Goals

Outcome 1 / Output 1.1

Increase software stewardship levels of our deployed code

Assess deployed code and prioritize stewardship gaps.

Dependencies on: Code Health Group

Goal(s)

Investigate and propose record of origin (ROO) for deployed code (currently Developers/Maintainers page)

Status

Note: July 2018

To do

Note: August 10, 2018

To do should get started on this in the next week or so (trying to get to a single record of origin (automated if possible))

Note: September 14, 2018

This is now

In progress and will need to do a few conversations with SRE folks for clarification and information for ROO.

Note: October 2,, 2018

This is now

Partially done and will wrap up in the next couple of days.

Outcome 1 / Output 1.3

Increase software stewardship levels of our deployed code

Using stewardship review process, create plan of action for top priority items each quarter.

Dependencies on: Development Teams

Goal(s)

Perform existing review process for Q1 cycle.

Status

Note: July 2018

To do

Note: August 10, 2018

To do this work is scheduled to start in the next week.

Note: September 14, 2018

There wasn't anything submitted so far this quarter, so it's still in a

To do status until there is work to be done.

Outcome 2 / Output 2.1

We reduce the number of testable regressions from hitting our users

Integrate regression testability evaluation into our on-going post-mortem process.

Dependencies on: Release Engineering (re: TEC3, Outcome 2)

Goal(s)

Add test evaluation to post mortem review process.

Status

Note: July 2018

To do

Note: August 10, 2018

In progress

Note: September 14, 2018

This is now

Done, we might need a slightly different way to collect this information, as right now the document is a bit difficult to use after an incident. We'll take a look at this in the future.

Outcome 2 / Output 2.2

We reduce the number of testable regressions from hitting our users

Jointly create smoke tests addressing high priority needs for 15 projects over the year.

Dependencies on: Development Teams

Goal(s)

Review existing e2e test coverage.
Define prioritization scheme.
Prioritize e2e testing gaps.

Status

Note: July 2018

To do

Note: August 10, 2018

To do Need to find time to work on this, but it's a high priority task

Note: September 14, 2018

This has been

In progress for the last couple of weeks — we're looking at the repos in production and noting the activity in the repos over time and will figure out which ones need the end to end coverage.

Note: October 2, 2018

This has nearly finished, and is

Partially done

Outcome 2 / Output 2.3

We reduce the number of testable regressions from hitting our users

Pro-actively add unit tests to MediaWiki core and deployed extensions.

Dependencies on: Development Teams, Code Health Group

Goal(s)

Make current unit testing coverage more visible by reporting out to Engineering Management.
(maybe, TBC) set coverage goals

Status

Note: July 2018

To do

Note: August 10, 2018

In progress The unit testing coverage reporting is in progress but the actual report hasn't yet gone out quite yet.

Note: September 14, 2018

This is still

In progress and will have the first report out by end of Sept.

Note: October 2, 2018

This is now

Done

Outcome 3 / Output 3.2

Reduce Technical Debt

Tech Debt Management process rolled out.

Dependencies on: Development Teams

Goal(s)

Platform and Search Platform teams are using TDM PoC

Status

Note: July 2018

To do

Note: August 10, 2018

In progress Worked with both Platform teams to talk about their tech debt and how to manage them. Might want to revise / change up the existing tech debt tags in Phabricator.

Note: September 14, 2018

This is currently

Stalled due to the other teams being super swamped. Will roll over to Q2's work.

Outcome 3 / Output 3.4

Reduce Technical Debt

Reduce technical debt in the MediaWiki core, by refactoring and improving internal interfaces and policies.

Dependencies on: Platform Team

Goal(s)

Identify key Tech Debt areas
Put in place Tech Debt management process for PEP

Status

Note: July 2018

To do

Note: August 10, 2018

In progress Part of output 3.2's work, going slowly because PE Platform is still getting organized.

Note: September 14, 2018

This is currently

Stalled due to the other teams being super swamped. Will roll over to Q2's work.

Outcome 4 / Output 4.1

Increase visibility into Code Health

Define Code Health Metrics

Dependencies on: Code Health Group

Goal(s)

Define base Code Health metric set.

Status

Note: July 2018

To do

Note: August 10, 2018

In progress Gotten skeleton crew up and running, getting base set of metrics identified.

Note: September 14, 2018

The working group has met for the first time, and have a follow-up meeting in another week or so. Everything is

In progress at this point and we might want to update the goal based on the working group's conversations to figure out 1 or 2 meaningful metrics.

Note: October 2, 2018

This is now

Done

Q2 Goals

Outcome 1 / Output 1.1

Increase software stewardship levels of our deployed code

Assess deployed code and prioritize stewardship gaps.

Goal(s)

Update/refresh review queue (review process for initial code deployment)

Status

Note: October 2, 2018

This work has not yet started

To do

Note: November 8, 2018

Initial proposal was created and now

In progress and waiting to discuss with stakeholders.

Note: December 6, 2018

This is still

In progress and we will meet up with SRE to go over initial proposal thoughts. This will continue in to Q3.

Outcome 2 / Output 2.1

We reduce the number of testable regressions from hitting our users

Integrate regression testability evaluation into our on-going post-mortem process.

Dependencies on: Release Engineering (re: TEC3, Outcome 2)

Goal(s)

none

Status

Note: October 2, 2018

This work has not yet started

To do - will work on it when there is work to do.

Note: November 8, 2018

No PMs thus far.

Note: December 6, 2018

This goal was adding to post mortem reports, and can be considered

Done

Outcome 2 / Output 2.2

We reduce the number of testable regressions from hitting our users

Jointly create smoke tests addressing high priority needs for 15 projects over the year.

Dependencies on: Release Engineering (re: TEC3, Outcome 2)

Goal(s)

5 of the 15 prioritized repositories have at least 1 end-to-end test.

Status

Note: October 2, 2018

This work has not yet started

To do

Note: November 8, 2018

Prioritization data has been collected and starting to reach out to top 5 candidate teams.

Note: December 6, 2018

We will contact stakeholders about creating tests for the prioritized repos, this goal is now

In progress.

Outcome 2 / Output 2.3

We reduce the number of testable regressions from hitting our users

Pro-actively add unit tests to MediaWiki core and deployed extensions.

Dependencies on: Development Teams, Core Platform Team, Code Health Group

Goal(s)

Assess Platform unit test practices and define improvement plan

Status

Note: October 2, 2018

This work has not yet started

To do

Note: November 8, 2018

This work has not yet started

To do

Note: December 6, 2018

We have now begun working with the Core Platform team on this

In progress

Outcome 3 / Output 3.2

Reduce Technical Debt

Tech Debt Management process rolled out.

Dependencies on: Development Teams, Core Platform Team, Search Team, Code Health Group

Goal(s)

Core Platform and Search Platform teams are using TDM PoC

Status

Note: October 2, 2018

This work has not yet started

To do

Note: November 8, 2018

Had initial discussions with Editor group to understand their current approach to managing tech debt.

Note: December 6, 2018

Met with the Core Platform team last week, much of the work currently

In progress with the Platform Evolution program, and is already oriented around this goal of tech debt.

Outcome 3 / Output 3.4

Reduce Technical Debt

Reduce technical debt in the MediaWiki core, by refactoring and improving internal interfaces and policies.

Dependencies on: Core Platform Team, Code Health Group

Goal(s)

Identify key Tech Debt areas
Put in place Tech Debt management process for PEP

Status

Note: October 2, 2018

This work has not yet started

To do

Note: November 8, 2018

This work has not yet started

To do

Note: December 6, 2018

Met with the Core Platform team last week, much of the work currently

In progress with the Platform Evolution program, and is already oriented around this goal of tech debt.

Outcome 4 / Output 4.1

Increase visibility into Code Health

Define Code Health Metrics

Dependencies on: Code Health Group

Goal(s)

Metrics defined and deployed for all 4 Code Health areas.

Status

Note: October 2, 2018

This work has not yet started

To do

Note: November 8, 2018

Code Health Metrics working group has been working steadily to review/assess code metrics tools. Current focus is on SonarQube. PHPMetrics has also been deployed for review purposes.

Note: December 6, 2018

We did an update in TEC3 for risk assessment; this goal has changed a bit this quarter, as we are now doing a proof of concept with SonarQube and is

Partially done

Q3 Goals

Outcome 1 / Output 1.1

Increase software stewardship levels of our deployed code

Assess deployed code and prioritize stewardship gaps.

Goal(s)

Develop and communicate guidelines and best practices for successful Code Stewardship.
(Continued from Q2) Update/refresh review queue (review process for initial code deployment)

Status

Note: January 10, 2019

Discussed that as we've just gotten back from our vacations, this work is ramping up and is In progress

Note: February 5, 2019

Have a basis for guidelines that should be published / reviewed in the coming weeks. Review queue work will start in another week or so.

Note: March 12, 2019

We are In progress and will wrap this up by end of this quarter (develop guidelines). Review queue work has gotten a bit bigger than expected and still In progress, we do hope to be done by end of quarter.

Note: April 12, 2019

The Code Stewardship Best Practices wiki page has been created with a skeleton of topics. This will be a living resource, but the work for Q3 is considered Done

Outcome 2 / Output 2.2

We reduce the number of testable regressions from hitting our users

Jointly create smoke tests addressing high priority needs for 15 projects over the year.

Dependencies on: Release Engineering (re: TEC3, Outcome 2)

Goal(s)

5 of the 15 prioritized repositories have at least 1 end-to-end simple sample test.

Status

Note: January 10, 2019

Discussed that as we've just gotten back from our vacations, this work is ramping up and is In progress

Note: February 2019

We've been reaching out to the top 5 emails/users for this goal, we're waiting on feedback from folks now that All Hands is over.

Note: March 12, 2019

This goal has now been indefinitely N Declined for a variety of reasons (possible death march)

Outcome 2 / Output 2.3

We reduce the number of testable regressions from hitting our users

Pro-actively add unit tests to MediaWiki core and deployed extensions.

Dependencies on: Development Teams, Core Platform Team, Code Health Group

Goal(s)

Evolve/develop tools and processes to support the PE refactoring effort to improve code health.
Develop common test strategy that enable teams to engage in more effective and efficient testing practices. (maybe should be output 2.4?)

Status

Note: January 10, 2019

Discussed that as we've just gotten back from our vacations, this work is ramping up and is In progress

Note: February 5, 2019

Discussed this with the CPT team and will meet again this week for figuring out next action items. We've had additional talks with test strategy but will continue next week as we're getting over wikiflu.

Note: March 12, 2019

Met last week with Core Platform team, to talk about addressing gaps and work is In progress - we're making good progress.
The rough draft outline is Done for the test strategy but we need to finalize a few things, so still overall In progress with expectations that we'll finish by end of quarter.

Note: April 12, 2019

Groundwork is set for this, we worked with CPT to define gaps and priorities and a course of action as they progress through their refactoring efforts. This will be a continued effort throughout the refactoring activities. Done
We'll need to do more discussion to develop a common test strategy, so this work will continue in Q4 In progress

Outcome 3 / Output 3.2

Reduce Technical Debt

Tech Debt Management process rolled out.

Goal(s)

Speak at All Hands on the status of Technical Debt
Engage and coach development teams on their approach to managing technical debt.

Status

Note: January 10, 2019

Discussed that as we've just gotten back from our vacations, this work is ramping up and is In progress

Note: February 5, 2019

The talk did not happen at All Hands, will look for another forum for this for a technical audience to review with.

Note: March 12, 2019

This goal has not been able to be completed and will probably be wrapped into the broader discussion of code health with various teams N Postponed. We hope to have a session at the Hackathon to talk about tech debt and code health.

Outcome 4 / Output 4.1

Increase visibility into Code Health

Define Code Health Metrics

Dependencies on: Code Health Group, Code Health Metric Working Group

Goal(s)

Code Health Dashboard with 50% of repositories covered.

Status

Note: January 10, 2019

Discussed that as we've just gotten back from our vacations, this work is ramping up and is In progress

Note: February 5, 2019

Made great progress on this during All Hands and will continue on it, should be on track for finishing it soon.

Note: March 12, 2019

We are on track to finish this in the quarter, In progress with the sonarqube integration.

Note: April 12, 2019

SonarQube is our tool of choice for our current set of code metric analysis and reporting. It's been integrated into the experimental pipeline and reporting for all extensions have been enabled. Done

Q4 Goals

Outcome 1+3

Outcome 1: Increase software stewardship levels of our deployed code
Outcome 3: Reduce Technical Debt

Goal(s)

Presentation/session(s) at the Wikimedia Hackathon on the current state of Code Health projects (technical debt and code stewardship)

Status

Note: April 8, 2019

This will happen in May 2019

Note: May 7, 2019

This will happen next week.

Note: June 4, 2019

Presentations were Done at the Hackathon and generated a lot of excitement

Outcome 1 / Output 1.1

Increase software stewardship levels of our deployed code

Assess deployed code and prioritize stewardship gaps.

Dependancies on: ___________

Goal(s)

Publish a re-imagination of the Review Queue process.
Develop and implement metrics around task and code-review responsiveness

Status

Note: April 8, 2019

This is also In progress

Note: May 7, 2019

Publishing a re-imagination of the Review Queue is N On hold right now
Metrics is still In progress

Note: June 4, 2019

Review Queue is still N On hold right now, blocked on more urgent work right now.
Metrics are still In progress - more work to come from the code review group and cross-team conversations. Work will continue in to next FY.

Outcome 4 / Output 4.2

Increase visibility into Code Health

Code Health Dashboard with 50% of repositories covered.

Dependancies on: _CI__________

Goal(s)

Expand SonarQube reporting into CI infrastructure
Perform SonarQube analysis on all extensions
Engage user communities in direct feedback solicitation

Status

Note: April 8, 2019

Expanding and performing the SonarQube is In progress and will engage the community later on in the quarter.

Note: May 7, 2019

Most of the patches are ready, just need to be reviewed, still In progress

Note: June 4, 2019

First pass was to create a pipeline and we then added in a lot of extensions (more than 50%); created code health pipeline - currently at 10 extensions and probably will add about 80 more. We think that the rest of the work will be done by the end of the quarter.