VisualEditor/Backlog design

We track our team backlog using the VisualEditor project in Phabricator. This page explains how exactly we structure it.

Tranches

We use open-scope tranches to group our upcoming work. The lower the tranche number, the higher its priority. Tranche 0 will never be closed (because there will always be bugs and breaking changes to our dependencies), but most of the others eventually will eventually close when the work is finished, or be de-prioritized once urgent milestones are met. Once that happens, we won't reuse tranche numbers, but instead create a tranche 5, tranche 6, and so on.

Last updated: 2017-01-03.

Current tranches

Tranche 0: Interrupt

Maintaining the production service at a suitable quality level:

No loss/corruption of existing article data
No loss/corruption of the user's contribution
No security issues
No regression in performance
No regression in usability

Tranche 1: Release support

Support community-requested releases/deployments, and urgent key usability improvements as they arise

Goal: Provide improvements to the Beta Feature of the wikitext edit mode — T142523

Goal: Provide a single edit tab interface, and switch all WMF wikis over to using it (without any other change in configuration) — T102398
Goal: Run an A/B test of providing the visual editor by default rather than the wikitext editor on clicking 'edit' for a small proportion of anonymous users on the English Wikipedia — T119269
Goal: Enable by default for all users on all Wikisources – T138966 for bugs, then T138391 for deployment

Tranche 6: Visual diffs

Provide an alternative to wikitext two-column diffs for users, in case they are not fluent in wikitext, and where it would be more appropriate. This covers:

Goal: Letting users choose between visual and wikitext views in the "Review your changes" tab inside the visual and new wikitext editors, on desktop and mobile — T143350
Goal: Letting users choose between visual and wikitext views in MediaWiki's diff page, on desktop and mobile — T105173
Exploring letting users "play" over the history of a page, seeing a stack of visual changes animate

Lower-priority tranches

Tranche 2: Mobile MVP

Improve VisualEditor for mobile tablets and phones to the point where it is credible to provide it by default

Goal: Support sub-documents inside the editor stack, and use this to support loading parts of the page asynchronously and only when needed — T76544
Goal: Load parts of the editor asynchronously and only when needed — T54365
Goal: Provide a way to edit parts of a page (paragraphs, tables, images) individually — T50429

Tranche 3: Language support

Expand VisualEditor's content language support

Goal: Enable by default for all users on the remaining, language-variant Wikipedias – T93388
- Support for "language conversion" blocks in Parsoid – T43716
- Design and provide an interface to editing "language conversion" blocks – T49411

Tranche 5: Rich media tools

Improve the editing of various 'rich' editing tools so that particularly-difficult tasks are simplified – specifically:

Images, including uploads (part of MediaWiki core),
Formulæ ("Math" extension),
Charts ("Graph" extension),
Galleries (part of MediaWiki core), and
Sheet music ("Score" extension).

Decision tree

Priority issues:

Functionality regression
Performance regression
Data corruption
Security issue
Crash of entire editor
Crash of bits of editor (e.g. copy & paste stops working)

Rating:

Very common -> Unbreak now! if a priority issue, Normal if not
Common -> High if a priority issue, Normal if not
Sometimes -> Normal if a priority issue, Low if not
Rare / very rare -> Low if a priority issue, Lowest if not

Features:

On list for this quarter -> High
On list for this year -> Normal
On list for next year -> Low
Not on list -> Lowest

Former tranches, now complete

Note: Former goals in active and former tranches are not listed for brevity.

Tranche 4: Link editor tweaks

Improve the link editing experience

Active for "FY2016Q2" (completed in October–December 2015).

Story points

We give each task story points as a very rough way of quantifying how much person-time we expect the task to take (including the time it takes to triage, investigate, and respond to the bug).

1 point: 1–3 person-hours
8 points: 1–3 person-days
40 points: 1–3 person-weeks
160 points: 1–3 person-months

Process

Make the minimum number of changes to the VE Phabricator data necessary to produce:
1. A stack-ranked backlog suitable for discussion and planning by most stakeholders
2. A Product Burnup chart
Don't (yet) try to do anything with orthogonal categorization of VE tasks (for example, don't try to tag all mobile tasks vs all desktop tasks, and/or all security vs all performance). May come back to this later but not needed yet.
Don't try to create a new project to contain a full stack-ranked list of all VE bugs (well, stack-ranked from 1-100).
1. Reason 1: huge amount of Phab spam if we bulk-move 1300+ active and 3000+ closed tasks.
2. Reason 2: Don't explicitly need it yet
Do create tranches in order to organize the backlog.
1. A tranche is an ordered set of all Stories and Epics (Tasks, in Phab) in a specific area of functionality that are largely independent of work in other tranches.
2. For now, each tranche is intended to address a group of related issues. (I.e., each tranche is effectively the top-priority tasks from a set of all tasks within a category, but the full list of all tasks in each category is not yet explicitly documented within Phab).
3. Tranches are implemented as columns within the VisualEditor project in Phabricator.
4. VisualEditor shall always have an open Tranche 0, defined as "Interrupt", tasks which are urgent enough to jump ahead of all other planned work.
  1. Tranche 0 will also be stack-ranked.
  2. Tranche 0 will include (among other things) critical bugs for Wikitext, VisualEditor, and any other projects the VE team is responsible for.
  3. Logically, therefore, work on Tranches 1+ will only occur when Tranche 0 is empty or when the VE team is sufficiently staffed to have capacity available after engaging with all Tranche 0 tasks.
5. All numbered tranches should be maintained fully stack-ranked - OBSOLETE, NEED TO UPDATE
6. TR0, and the next two active Tranches should be carefully ranked; other tranches may be loosely ranked. - OBSOLETE, NEED TO UPDATE
7. As a tranche is completed, it is hidden but not deleted/emptied. So Tranche 1 will disappear soon, and eventually "Tranche 20" will be the new "Tranche 1". To be clearer, we will use "Interrupt Tranche" for Tranche 0, "Primary Tranche" for the next open Tranche, "Secondary Tranche" the one after that, etc.
8. There is no intentional ranking for any tasks that are not in Tranches
Within tranches, work is grouped by Milestones - INCOMPLETE, NEED TO UPDATE
The existing VisualEditor columns (of the form "Next Up:" in Phabricator) may serve as a card mapping area for items before they are assigned up to a tranche or down to the unordered lower backlog.
1. This area may come and go. Currently, we don't see a need for it in this SOP.
The complete, stack-ranked list of VisualEditor tasks may therefore be derived from this algorithm: - OBSOLETE, NEED TO UPDATE
1. All open tasks in Tranche (0), in "natural order" as stored on the Phabricator board
2. All open tasks in Tranche 1, in natural order
3. All open tasks in Tranche 2, in natural order.
4. Etc through all numbered tranches.
  1. Note that technically, Tranche 1 is still more important than Tranche 20 even when tranches 1-19 are all Done and closed and hidden. This seems purely theoretical but may matter for reporting.
5. All open tasks in any "Next Up" columns, in no order (no order with columns and no order between columns)
6. All open tasks in To Triage and in Backlog, in no order
We will try to construct a Product burnup from a Phab data dump of this arrangement and refine the plan if that is not possible

Use Cases

Developer: Use backlog to determine next work to do
Product Owner: Prioritize work to meet goals
Any Stakeholders: view burnup to assess capability, work load, and forecast times for team work.
Active Stakeholders: view backlog identify work coming up and plan around it. (via roadmap?)
Goals-based burnup, to compare with tranche-based burnup, to see if they align
For gap analysis of goals vs tasks
1. Goals with no task:
  1. Should we be doing more to accomplish this goal?
  2. Should we cut this goal?
2. Tasks with no goal:
  1. Should we be doing this task? At this priority?
  2. Should we have a new goal (make explicit a goal that is currently implicit)?
  3. If so, what should its priority be, and how does that affect resourcing?
Any stakeholder: use a URL to have permanent reference to work requested or planned by the Foundation or movement.

Open questions

How far can an Epic go before it must be decomposed?
- Option: No epics in TR0, Primary Tranche, Secondary Tranche. <-- Decided
- After an Epic is broken out, the Epic tag should be removed and it should become either closed (since it's wholly juiced out) or a Tracker (if it retains some integration work, which must still be <= 1 task worth of work)
What's the definition of an Epic for VE?
- Will take one member of VE staff more than a few days of solid work. In practice, over 8 points. Decided.
How is pre-coding work (requirements, design) accounted for?
- Option A: Part of continuous "Backlog Grooming" that is assumed to take a fixed percentage of Product Owner's time as overhead. Not counted in points.
- Option B: separate "Grooming" story for preparing one or more other stories to be shovel-ready. Pointed and forecast.
- Option C: considered part of the lifecycle of a story, so that a story should be created as early as possible after basic scope definition, and the story assigned to appropriate people in sequence (such as designer, User Research, etc) and pointed and forecast appropriately. <-- Decided
  - Some of option A is inevitable; e.g., bulk work to prioritize or define 50 tasks in one 30-minute sitting is not pointed or tracked.
  - If anyone non-Product Person (James or Neil) is spending time grooming or working on a Task (good rubric: is it assigned to them?), it should be a Task or part of another Task, be pointed, and be estimated.
- Option D: Spikes, which are unpointed stories that are identified and prioritized, but not estimated or forecasted.
What are we doing with the Doing column?
- keep it? Then how does it relate to tranches?
- move those tasks back into Tranches?
- Treat that as tranche -1, and then stop using it after current tasks are done <-- Decided
What are we doing with the Done column?
1. stop using it after current tasks are done <-- Decided
What is an activity period? A monthish? (Decide Later)
1. Don't need to nail this down - will be informed by data.
How much of the backlog has to be well-ordered to have a realistic plan 4-6 months out? (Decide Later)
1. Guessing 100 items.
2. As of 18 June 2015, there are about 50 fully stack-ranked items and 1300+ tied for 51st.
3. Will come back to this in a few weeks or months.
Can/should/will we do any ordering outside of the top hundred (or fifty)?
1. Option A: use vague tranches
2. Option B: use "Next Up" columns to hold things above the 1300+ but below the numbered tranches
3. Option C: Use priority field
4. Option D: actually stack-rank (where - in what Projectcolumn? could do this in Backlog column in VisualEditor)
5. Option E: Like Option B but with Epics and Super-Epics
What does the Priority field mean for VE tasks?
1. Option A: use the priority field for priority (and if so, how will it interact with the stack-ranking)
2. Option B: severity as perceived by reporter <-- Decided
3. Option C: something else
4. Option D: ignore it

Goals

What exactly is “goals”?
1. Objective + Key Result(s)
What is the relationship between milestones and KRs?
1. Option A: Milestone is another kind of Key Result
2. Option B: Milestone is a combination of Key Results
3. Option C: Milestone is a combination of
What are the useful queries for different stakeholder groups, and does this design support them?
1. See Use Cases above.
What are the edge cases? E.g., Tasks which are technically small but have complex implications or dependencies - treat as Task and pass-through Epic, or Task is Epic, or Task rolls up into catch-all Epic?
How exactly do we get the roadmap back from this?
1. Hierarchical Backlog: List of Epics
2. Heterogeneous backlog: ?
How do we relate OKRs to tasks?
1. Leave OKR in Wiki pages, don't link tasks to OKRs
2. Leave objective in wiki page, create a tranche projectcolumn for each key result
  1. Put tasks into the related tranche
3. Leave objective in wiki page, create a task for each KR
  1. Make related tasks blockers of the KR tracking task
4. Create task for each objective and create subtasks for each tracking task
  1. Make related tasks blockers of the KR tracking task
  2. Have to retain a usable presentation on the wiki

Estimation

How are Epics estimated?
- Option A: Use "Epic Points", either 1/2/4/8/16 or Fibonacci, which are separate from Story Points but calibrated to them
- Option B: Use very big numbers of Story Points <-- Decided
- Option C: Use T-shirt sizing (S/M/L/XL), correlate to Story Points via calibration sample
How far through the backlog should we estimate?
1. Option A: All stack-ranked <-- Decided (need to try for 1-4 weeks and see if this satisfies Option E)
2. Option B: All stack-ranked plus all Next Ups
3. Option C: All stack-ranked, and others as touched/needed
4. Option D: all 1300 :)
5. Option E: Enough to get 3-5 months' visibility
who does it for initial catchup? James, Done.
Who does it on an ongoing basis?
1. James
2. Ed
3. Neil
4. Rotating ([1])
5. Some two-level pass of initial + validation
Are there homogeneous groups of tasks that can be assigned the average estimate from a subset?
1. The full 1300, based on average of all estimated tasks (including history, not including history?
2. No <-- Decided
3. some other set
What range of values should we use for velocity? (Decide Later)
1. come back to this