Extension:WikiLambda/Metrics

This page describes the metrics that are currently collected from the WikiLambda extension, and the dashboards on which most of these metrics may be viewed. These metrics are collected to provide information about the level of usage of Wikifunctions and its various features, and the degree of success or difficulty that users may have in working with the different features, which in turn can provide valuable guidance for the further evolution of Wikifunctions.

The datasets from which these metrics are computed are not made public. Because these datasets contain a few types of information that are personally identifiable, such as SUL usernames and device identifiers, they are carefully protected and accessible only to individuals with appropriate permissions. If and when datasets are shared, they will be anonymized; that is, personally identifiable information will be omitted.

Links to relevant Phabricator tickets are given below (e.g. T340786) for most of the metrics categories.

Metrics Currently Collected

Interaction Metrics

These metrics count up specific types of interactions in the Wikifunctions user interface. They are organized here by Wikifunctions’ Critical User Journeys (such as Evaluate a function and Create a function definition).

1. Evaluate a function (T340786)

• Number of clicks on Run Function button
  • Breakdown by page:
    • Function page
    • Implementation page
    • Special evaluation function page
    • Test page
• Function call completion rate (the first divided by the second)
  • Number of successful calls to functions
  • Number of all calls to functions
• “Function popularity”: How many times each called function gets called

2. Create a function definition (T340160)

• Number of function-creation attempts started
• Number of function-creation attempts ending with Publish
• Number of function-creation attempts ending with Cancel

3. Edit an existing function definition (T340779)

• Number of function-edit attempts started
• Number of function-edit attempts ending with Publish
• Number of function-edit attempts ending with Cancel

4. Create a test (T340783)

• Number of test-creation attempts started
• Number of test-creation attempts ending with Publish
• Number of test-creation attempts ending with Cancel

5. Edit an existing test (T348285)

• Number of test-edit attempts started
• Number of test-edit attempts ending with Publish
• Number of test-edit attempts ending with Cancel

6. Create an implementation (T340784)

• Number of implementation-creation attempts started
• Number of implementation-creation attempts ending with Publish
  • Number of these using code
  • Number of these using composition
• Number of implementation-creation attempts ending with Cancel
  • Number of these using code
  • Number of these using composition

7. Edit an existing implementation (T348285)

• Number of implementation-edit attempts started
• Number of implementation-edit attempts ending with Publish
  • Number of these using code
  • Number of these using composition
• Number of implementation-edit attempts ending with Cancel
  • Number of these using code
  • Number of these using composition

Product Metrics

Content (T341748, T341733, T341734)

• Number of functions present
• Number of implementations present
• Number of connected implementations
• Number of tests present
• Number of connected tests

Performance

• Pageload time (patch 924177)
• Calculations:
  • Average for all pages
  • Average for each type of view (currently 4 distinct types)
  • Averages for viewing existing objects and for editing new objects

User / Usage Metrics

Adoption

• Number of new users
• Number of active users
• Number of users whose accounts become inactive
• Number of unique devices accessing Wikifunctions

Community diversity

• Number of different languages used in Wikifunctions
  • [Languages with at least one label on at least one Function]
• Number of different countries represented by users
• Number of (new, active) users per country

Dashboards

Most of the metrics listed above may be viewed on either the Wikifunctions Dashboard (Wikifunctions top-level tracker) or Wikimedia Statistics. In accord with Wikimedia Foundation privacy policies, the Wikifunctions Dashboard is only accessible to those who have been granted appropriate permissions.

• The Interaction Metrics are located on the Interaction metrics tab of the Wikifunctions Dashboard.
• The Product Metrics are located on the Top-level metrics tab (2nd section) of the Wikifunctions Dashboard.
• The User/Usage Metrics are partly located on the Top-level metrics tab (Sections 1, 3, and 4) of the Wikifunctions Dashboard, and partly located on Wikimedia Statistics.

Wikimedia Statistics also includes some other user / usage metrics not listed above.