Contributors/Metrics
 | This page is obsolete. It is being retained for archival purposes. It may document extensions or features that are obsolete and/or no longer supported. Do not rely on the information here being up-to-date. |
Like any team, the Editing department will do better if we have a set of clear, well-thought-out performance indicators that we can use to evaluate our work.
Key metrics
[edit]All these metrics are current calculated using manual queries on our editor month table. Historical values are available in an online spreadsheet.
| Phenomenon | Metric(s) and rationale | Definition | Rationale |
|---|---|---|---|
| Community size | Active editors | Number of registered users who made at least 5 edits in content namespaces across all Wikimedia projects in a given month. (more,see also) | The standard, Foundation-wide barometer for the health of the contributor community. |
| New editor intake | New active editors | The subset of active editors in a given month who signed up for their account in that same month. (more) | |
| Editing activity | Non-bot edits | The number of edits made to all namespaces on all Wikimedia projects by users not identified as bots. | |
| Mobile editing activity | Mobile edits | The subset of non-bot edits which are tagged as made through either a mobile web or mobile app interface. | Tracks the success of mobile editing, a Foundation priority. |
Consumers
[edit]- WMF quarterly reports
- Editing team quarterly reviews
- Audiences updates page
- Wikistats 2.0?
Considerations
[edit]Some of these metrics, like global active editors and global mobile edits, are mostly out of our control. If we see, say, the global active editors number trend upwards over 6 months around the time we roll out a headline new feature, we have very little way to know if the new feature was responsible and many reasons to suspect that it wasn't (because so many other thing influence the metric). Whether they go up or down or stay flat, such metrics are interesting but not actionable. As Aaron Halfaker puts it, we might build a great windmill and write it off as a failure because the wind isn't blowing (or the reverse).
So, these numbers have limitations as metrics for our department. However, there's no question that it's highly important for the WMF and for the movement as a whole to have these numbers, and it doesn't seem that there's any better team to take responsibility for them than us.
However, there may still be opportunities to pick better metrics within these constraints. For example, focusing on rates (like the rate at which editors who've registered become contributors) instead of absolute numbers could take the high level "winds" (e.g. people's interest in Wikimedia projects) as given and focuses instead on how efficiently we convert them into our desired outcomes.
In addition, Editing contains a fairly heterogeneous group of teams. It not clear that one metric, or four, can be an actionable guide for all of them at once.
Barometer projects
[edit]Sometimes it's necessary or desirable to track individual projects rather than global numbers. However, it's always impossible to track hundreds of projects individually, so it's helpful to have a list of specific projects which we focus on. There are a number of considerations involved:
- Large size, which makes the data less noisy.
- Mix of mature and developing projects.
- Global diversity
- Mix of Latin and non-Latin scripts
One very simple solution is the six Wikipedias in the official languages of the United Nation.
List, plus May 2015 active contributors and rank thereby:
- English Wikipedia (31 601, 1st)
- French Wikipedia (4 602, 3rd)
- Spanish Wikipedia (4 318, 4th)
- Russian Wikipedia (3 315, 6th)
- Chinese Wikipedia (2 378, 8th)
- Arabic Wikipedia (944, 12th)
- Should we also include Commons? It's not clear that most metrics will apply well to it.
Other possible metrics
[edit]| Metric | Notes | Breakdowns |
|---|---|---|
| Editing activity | Labor hours based on session duration | |
| New editor activation rate | Almost all of the change seems to be driven by changes in registrations, not in new active editors. The absolute number of new active editors would avoid this problem. | |
| Reactivated editors | A component of active editors. | Project |
| Recurring editors | A component of active editors. | Project |
| Surviving new editors | A component of active editors. | Project |
| Surviving pages created | New pages in main namespace not deleted within 48 hours (not long enough for AfD or prod on en.wiki)? | Project |
| Surviving uploads | Uploads (to Commons only? would exclude fair use images, etc.) not deleted within 48 hours? | |
| Content richness index? | Density of desirable features like (1) citations, (2) wikilinks, (3) media, and (4) templates.
Denominator:
|
Project |
Metrics selected by others
[edit]Apr–Jun 2015 quarterly report
[edit]Here.
- Participation
- Sign-ups
- New editors
- Active editors
- Content
- New articles
- Edits