Documentation/Tools/Documentation metrics dashboard
This page contains requirements for the documentation metrics dashboard. The first version of the dashboard was released on 30 July 2024. For more information, see task T334839. For initial design ideas, see Documentation metrics dashboard/Design.
Rationale
[edit]Technical writers are missing a single source of relevant information about the ecosystem of technical documentation. This makes it difficult to effectively plan work that focuses on anything more than a few pages at a time. Reasoning about documentation collections, or technical documentation as a whole, is pretty much impossible.
Goal
[edit]Provide staff and community members with access to statistical data about technical documentation pages and collections they're interested in.
Predicted result
[edit]Establish a baseline of data to collect about single pages and collections of pages, and then present that data on a publicly available dashboard. For MVP, the dashboard tracks readership and edits, Phabricator mentions, and other details, ensuring that all insights relevant to stewardship of technical documentation are always easy to access.
Data collection
[edit]The purpose of this project isn't to introduce any new tracking, but to collect the already available metrics in a single place for ease of use.
Open questions
[edit]- How to deal with page translations? Specifically, should we treat translations of documentation pages as the same page or different pages?
Glossary
[edit]PagePile
[edit]PagePile is a service that allows you to create lists of wiki pages. It's available on https://pagepile.toolforge.org. Each list, a page pile, has a unique identifier.
Instead of providing an interface to specify lists of pages directly, the current version of the dashboard uses PagePile as a way of defining collections.
A rewrite of PagePile - GULP - is underway, with the new version available on https://gulp.toolforge.org. The plan is to support GULP in a later version of the dashboard.
Throughout this document I refer to the service as PagePile, and refer to individual lists available on PagePile as page piles.
Application, dashboard
[edit]I use the terms "application" and "dashboard" interchangeably to refer to the documentation metrics dashboard we are building.
Requirements
[edit]Functional requirements (MVP)
[edit]Listing page pile contents
[edit]As a user
I want to specify a page pile ID and view the pile's contents before opening the dashboard
So that I can ensure I'm requesting the dashboard for the correct list
Given that I am a user on the application's landing page When I provide a valid page pile ID with pages from tech wiki, meta wiki, or mediawiki.org And I click Load Then I will see the list of pages from that page pile
Given that I am a user on the application's landing page When I provide an ID of a page pile that doesn't exist And I click Load Then I will see an error message that explains this page pile doesn't exist
Given that I am a user on the application's landing page When I provide an ID of a page pile that exists but contains pages from a wiki other than tech wiki, mediawiki, or meta wiki And I click Load Then I will see an error message that explains this page pile is unsupported
Page pile history
[edit]As a user
I want to see a list of page piles I accessed previously
So that I can quickly find information about collections interesting to me
Given that I have explicitly set the Remember my page piles setting And I have consented to the application's usage of cookies and local storage When I successfully load content of a valid page pile Then this page pile is added to my page pile history
Given that I am a user who previously used the application in the same browser And I have explicitly set the Remember my page piles setting and consented to the applications' usage of cookies and local storage When I open the metrics dashboard Then I will see the list of collections I opened previously
Opening the dashboard
[edit]As a user
I want to access the statistics dashboard for pages in the page pile I requested
So that I can view the relevant page statistics
Given that I have loaded pages from a valid page pile And I am seeing the list of pages from that page pile When I click Open page pile statistics Then I will see the dashboard for that page pile
Dashboard - default view
[edit]As a user
I want to see aggregated information about an entire page pile based on information about individual pages
So that I can make high-level decisions about work needed to improve specific collections
Given that I am a user with the dashboard for a page pile open When I scroll down the page Then I will see different representations of aggregated documentation metrics
As a user
I want to see a list of pages in a page pile
So that I can view the statistics for a single page instead of the entire page pile
Given that I am a user with the dashboard for a page pile open When I click a specific page in that page pile Then I will instead see the statistics for that page only
As a user
I want to sort pages listed on the dashboard based on statistics
So that I can find pages where my contributions could have the highest impact
Dashboard - page view
[edit]As a user
I want to see the statistics for a single page instead of the entire page pile
So that I can understand which pages require documentation work
Dashboard - available metrics
[edit]As a user
I want to have access the following information on the dashboard:
- Number of page views, breakdown by day - in graphical or numerical form
- Number of page edits, breakdown by day - in graphical or numerical form
- Number of major edits vs minor edits - in graphical or numerical form, number of small (<1000b) and big (>=1000b) edits
- List of most active contributors with their statistics
- List and number of Phabricator tasks that mention the page
- General page information, page categories, incoming and outgoing links (only for individual pages), numbers of incoming and outgoing links (aggregated for the entire page pile), information about whether the page is marked for translation (and the percentage of pages in a page pile that are marked for translation).
- Indication whether a page uses specific templates, such as "Draft", "DoNotTranslate", "Outdated", or "Historical".
So that I can reason about the quality of pages and collections
Cookies or local storage usage
[edit]As a user
I want to see a blocking cookie or local storage popup or modal window when I attempt to use any application feature that uses cookies or local storage
So that I can grant the application my explicit consent to store data on my device
Functional requirements (later)
[edit]This sections describes functional requirements that we plan to meet in a later version of the dashboard.
Support for GULP
[edit]As a user
I want to provide a GULP list number
So that I can see the statistics dashboard for the collection represented by that list
Support for manually constructed lists and doc.wikimedia.org
[edit]As a user
I want to manually specify a list of valid pages from doc.wikimedia.org, tech wiki, mediawiki.org, and meta wiki to include in a collection
So that I can see statistics for more complicated page lists
This feature will allow us to construct lists with valid pages from multiple wikis and doc.wikimedia.org. PagePile only support lists with pages from a single wiki.
Note that some statistics might not be available for doc.wikimedia.org pages. We need to consider and implement this carefully so that we don't undermine the accuracy of our data.
Sharing custom lists
[edit](depends on Support for manually constructed lists)
As a user
I want to share my custom list using a simple code or number
So that I can help other users gain insight about lists of pages I have already built
Custom collections as a tree or table of contents
[edit](depends on Support for manually constructed lists)
As a user
I want to see the hierarchy of pages within a collection created directly in the application represented as a tree instead of a list
So that I and others can reason about the information architecture of the collection
Flat lists have limited use for technical writers. Trees are better at representing hierarchies of information. While this will not impact the statistics, having access to hierarchies of information in the form of trees will definitely help us reason about collections.
Linter integration
[edit]As a user
I want to have access to linter output for a specific page on the dashboard
So that I can see content change recommendations based on commonly used style guides
To implement this using vale.sh, we would need to create and maintain support for wikitext syntax ourselves (see https://vale.sh/docs/topics/scoping/#formats), or come up with a workaround.
Export to JSON
[edit]As a user
I want to be able to export dashboard information to a well-structured JSON file
So that I can use and process the data in another tool
Dashboard - additional metrics
[edit]As a user
I want to have access to the following additional information on the dashboard:
- Number and percentage of pages that have been translated. Number of translation pages for each page.
So that I can reason about the quality of pages and collections and consider translation impacts in my work
Non-functional requirements
[edit]Public access
[edit]The application is publicly available on the internet, under a URL that's easy to use and remember.
Personal preferences
[edit]The application doesn't require login and supports only one type of user - an anonymous user. Any user preferences and lists history are stored either in a cookie or in local storage (subject to user’s consent).
Localization
[edit]The application supports localization and allows users to set their preferred interface language. This setting is stored in their cookies or local storage if they have consented to their usage.
The application supports both LTR and RTL languages.
Inclusive design
[edit]The governing design principle is: "The application is easy to see, easy to hear, easy to interact with, and easy to understand".
The application allows users to set their preferred interface font to a dyslexia-friendly typeface.
The application's design follows the principles of neurodiversity design available on https://neurodiversity.design.
The application is designed from ground up to use all the modern accessibility features, with WCAG being the baseline.
The application is designed to minimize the CPU and RAM requirements of a device it is used on.
The application is designed to work well on internet connections with lower bandwidth and worse connection stability.
The application aims to serve users at the 75th percentile of devices and networks.
Caching
[edit]The application uses caching to minimize the number of requests to external APIs. Consecutive requests for the same data do not result in consecutive calls to these APIs.
Non-requirements
[edit]This section describes what we've decided not to implement.
User login and management
[edit]The dashboard does not provide any mechanisms of user management, login, or registration. Any preferences, custom data, and (eventually) manually-created lists are stored locally and are visible only to their creator.
Content moderation
[edit]The dashboard requires no moderation as it doesn't allow users to edit any publicly-visible data. Any user-created data is only visible to them.
Usage instructions
[edit]- Navigate to the application’s home page: https://techdoc-dashboard.toolforge.org/.
- In the PagePile ID field, specify the identifier of the page pile you want to analyze and click Load.
- The application displays the list of pages in the page pile. Click Open page pile statistics to proceed with the loaded pile, or specify a different PagePile ID and click Load again.
- NOTE: The application limits the number of pages to 100, regardless of how many pages are in the page pile. (phab:T378644).
- (Optional) Click Remember my page piles and consent to the use of cookies and local storage if you want to enable page pile history on the landing page.
- Clicking Open page pile statistics opens the statistics dashboard for the page pile. Scroll down the page to analyze the documentation metrics.
- The dashboard features a table with pages in the page pile. You can sort that table by some metrics. To view statistics for a specific page, click that page.
To access your preferences, click Preferences. Note that preferences work if you have consented to the application's usage of cookies and local storage.