Phlogiston/Configuring Phabricator to be reportable

Making Phabricator tasks reportable in Phlogiston means two things: there must be a way within Phabricator to identify all of the tasks within scope for a report (e.g., all of the tasks belonging to a team), and there must be a way within Phabricator to designate which tasks belong to which categories. Phlogiston can usually create a report from a team's existing Phabricator setup without any changes, but because most teams group work only by status, not by category, this initial report will be less valuable compared to reports that can be grouped by category.

What is the scope of the team's work?

How does your team identify its work? Phlogiston will need a list of Phabricator projects that include all of the tasks that your team works on, and all of the tasks that you consider to be part of your backlog, and no other tasks. Especially, no other tasks that other teams work on.

How do you identify categories of work?

Most teams group their work in Phabricator in terms of sequence of work, such as tasks to be designed, tasks to be tested, tasks ready to be released. For purposes of reporting and for answering the kinds of questions that Phlogiston reports answer, however, it is generally much more useful to group work in terms of purpose: goals, milestones, fixed-scope projects, ongoing projects, etc. Phlogiston supports several different approaches in Phabricator to grouping work.

Group by Project

This is the simplest: all tasks with a particular project tag can be grouped into one category in Phlogiston. No further configuration is required in Phabricator.

One example configuration is:

• All work that the team does is tagged with a project tag, such as Branding Team. This could also be a fixed-scope tag, such as Branding Team 2015Q4.
• The columns of the workboard for the Branding Team 2015Q4 project mark the state of the work, such as Proposed, Approved for Design, Being Tested.
• Different categories of work, such as Wikipedia 30th Anniversary and Copyright Symbols, each have their own Phabricator project tag. These tags are visible within the task cards on the Branding Team 2015Q4 workboard.

In this configuration, the only detail relevant to Phlogiston reporting is the last bullet; Wikipedia 30th Anniversary is a Phabricator project, and all tasks that belong to that project could be grouped into, for example, a 30th Anniv category in Phlogiston reports. (Category names in Phlogiston reports can be customized and do not have to exactly match the Phabricator names.)

Group by Project Column

All tasks in a particular column in a particular project can be grouped.

Group by Parent Task

All tasks that are "children" of a parent task, meaning they block the parent task, can be grouped. This works for all levels of a blocking relationship: tasks that block tasks that block a particular task can be grouped under that task.

Some teams already configure their tasks this way, making this a natural fit for Phlogiston reporting. However, because there is no easy way in Phabricator to see if a task has Category parent task, it is very difficult to maintain this data perfectly in Phabricator. Additional overhead, such as regularly running a Phlogiston report to see if there are any uncategorized tasks, is required.

For this to work in Phlogiston:

• The parent task must be tagged with the Category project tag. This tag is hard-coded into Phlogiston to denote tasks whose descendents constitute a category.
• All of the tasks in the family tree must be included in projects included in the Phlogiston scope.
  • For example, if the parent task is in the Phabricator project Widgets and has a sub-task in the project Unicode Widgets, and the sub-task has a sub-sub-task in the project Widgets, and the project Unicode Widgets is not included in the list of projects to be reported on but Widgets is, then the sub-sub-task would be included in the report but would not be recognized as a sub-sub-task of the parent task, because the intermediate task is not present in the report data.

Group by Intersection of two Projects

EXPERIMENTAL. Phlogiston can group all tasks that belong to two specific projects into a single category.

Example configuration

• All work that the team does is tagged with Branding Team 2015Q4.
• The columns of the workboard for the Branding Team 2015Q4 project mark the state of the work, such as Proposed, Approved for Design, Being Tested.
• Different categories of work, such as Wikipedia 30th Anniversary and Copyright Symbols, each have their own Phabricator project tag. These tags are visible within the task cards on the Branding Team 2015Q4 workboard.
• Copyright Symbols also includes many other tasks that other teams are working on, and which should not be included in Phlogiston reports. These tasks are not on the Branding Team 2015Q4 board.
• Phlogiston counts all tasks that have both the Branding Team 2015Q4 and the Copyright Symbols project tags as the reporting category Branding Team 2015Q4 Copyright.

Using heterogeneous categorization rules

A Phabricator configuration can use more than one of these categorization rules simultaneously. Phlogiston applies the categorization rules in a fixed sequence, so that all tasks that are not categorized by the first rule are available to the second and following rules.

Example:

• All work that the team does is tagged with Branding Team
• The columns of the Branding Team board correspond to different long-term initiatives, such as Anniversaries and Copyright and Holidays
• Within each column, specific milestones are denoted with tracking tasks such as 15th Anniversary and 20th Anniversary and Mayday 2020. These tasks are tagged as Category and all of the tasks related to them are marked as blocking them.
• With this Phabricator configuration, Phlogiston could categorize tasks from most to least specific, such as 15th Ann., 20th Ann., Other Anniversary, Copyright, Mayday 2020, Other Holidays, Other.

How to debug categorization rules

1. Set up Phabricator according to the above instructions.
2. Wait for the nightly data dump, and then run Phlogiston.
3. Look at the debugging reports linked from the top right part of the Phlogiston report page.
   1. Full List of Possible Categories. Example. This report shows all of the information about tasks in Phabricator that Phlogiston can use to categorize tasks. If Phabricator is configured with one of the ways described above to categorize a task, and there are tasks in that category, then the Phabricator category titles (e.g., the Project title, Project Column title, etc) should be present in this report. If it is not, then that category will be empty in Phlogiston.
   2. Rules for recategorizing tasks. Example. This is a link to the github file that defines the categorization rules for this report.
   3. Recently Closed Tasks. Example. This is list of all tasks resolved in the last 2 weeks. It should include all of the tasks that Phlogiston was told to track that were changed from open to resolved. It therefore can be used to confirm that Phlogiston is seeing what the Product Owner thinks it should. It can also be used to identify tasks that did not get categorized correctly, to ensure that there are no uncategorized resolved tasks in the data.
   4. All open tasks, sorted by category. Example. This is a list of all open tasks in this scope, grouped by category. It can also be used to figure out if Phlogiston is categorizing tasks as expected.
   5. Histogram of tasks by points. Example. This shows patterns in how a team has been assigning point estimates to their tasks. It is a histogram of the point value of all resolved tasks, grouped by Phabricator Priority field. Each column is a different point value. So the highest bar in each row is the most common point value for tasks in that row. Each row is a different Phabricator priority, From Lowest priority at the top to Unbreak Now! at the bottom. (10 = lowest, 25= low, 50 = Normal, 80 = High, 90 = Untriaged, 100 = Unbreak Now!). The last row is a summary of all tasks by point, so the highest bar in the bottom row is the most common (modal) point value for all tasks in this report.
      1. If a report includes a mix of pointed and unpointed stories, it may be helpful to set a default point value for unpointed tasks. Otherwise, the points-based reports may be misleading because they will not show the unpointed stories. Phlogiston will assign this point value to all stories with no points; it will not change stories with zero points. If the histogram shows that most tasks get the same point value, that would be a good default value.
      2. The same line in Phlogiston that links to this report also shows the default value that is currently set in the configuration file for this report.