Analytics/Research and Data/2014-15 Goals

see Wikimedia_Engineering/2014-15_Goals#Research_and_Data

Q1: Jul - Sep 2014

Primary goal

standardize baseline metrics

• define key organizational metrics (aka the Editor Model) and apply them to product team goals/instrumentation:
  • Growth
    • #501 – gather baselines for active editor breakdown  Done
    • #511 - choose numeric targets for 2014-15  Done
  • Mobile
    • #343 - support mobile metric goal setting  Done
    • #409 New Active Editor / Surviving New Active Editor baselines for mobile  Done
  • VE
  • MultiMedia
• define key project-level participation and content metrics:
  • Daily unique editors
    • #407 - Daily unique registered editors  Done
    • #408 - Daily unique anonymous editors  Done
    • #415 - Daily unique editing bots  Done
  • #416 Daily unique page creators (no breakdown)  Done
  • #417 Daily unique media creators (no breakdown)  Done
  • #423 - Daily edits  Done
    • #418 - Daily edits by registered users  Done
    • #420 - Daily edits by anonymous users  Done
    • #419 - Daily bot edits  Done
  • #421 - Daily pages created (no breakdown)  Done
  • #422 - Daily media created (no breakdown)  Done
• make significant progress on reader metric definitions:
  • Page views
    • #288 Pageview definition
  • Unique clients
    • #373 - LUCID scheme  Done

(italicized items are stretch goals as they have dependencies that we'll be able to assess throughout Q1, see the full list of metrics on Meta)

Secondary goals

topical research

• anonymous acquisition (Growth)
  • #465 Anonymous A/B test #1  Done
  • #559 Anonymous A/B test #2  Done
• Task suggestions (Growth)
  • #563 Qualitative similarity of morelike  Done
  • #573 Research questions for task recommendations  Done
• mobile app adoption/traffic and participation trends (Mobile)
  • #375 Mobile trends @ Monthly Metrics  Done
  • #294 Mobile trends report on Meta  Done
  • #379 Mobile app traffic analytics  Done
• predictive models of editor activation/survival
  • #344 Update revert data for enwiki @ beyond
    •  Done for enwiki

Staffing and team process

process & strategy

• uplevel team process (prioritization sprints, project management)
• finetune operating model (consulting vs embedded)
• push on horizontal integration with other research teams (UX, grantmaking)
  • #158 research touchpoints  Done

staffing

• F/T fundraising research analyst on-board (new hire)
  • #386 FR research analyst onboarding  Done
• start recruiting for traffic research analyst position
• aggregation/anonymization of traffic data (external collaboration)
• topic extraction (contractor, depending on availability)

Q2: Oct - Dec 2014

Top Q2 departmental priorities

* Increase mobile web contributions and mobile app readership
** Make editing faster

Readership *

• Finalize the new pageview definition, incorporating feedback from the first draft
• Deliver a unique client definition and implementation proposal
• Generate reports on readership trends from historical data, based on the new pageview definitions
• Generate recurrent reports of session-based metrics (uniques, pv/sessions, sessions/user) for the Mobile Apps team
• Start recruiting traffic researcher

Mobile microcontributions *

• Instrumentation, experimental design and evaluation strategy for mobile microcontributions
• Analysis of data collected from the first rollout of microcontributions on the beta site
• Background analysis of knowledge gaps suitable for microcontributions

Performance **

• Run a test on the impact of HHVM on newcomer engagement
• Analyze the performance of editing interfaces

Fundraising

• Support the team with the annual fundraiser push

Formal collaborations

• start formal collaborations with external researchers under NDAs

Analytics Dev support

• provide breakdown definitions (method/target site) for Vital Signs metrics

Product consulting

• support other product areas (pending prioritization)

Q3: Jan - Mar 2015

Top Q3 departmental priorities

* Release and test two new reader-targeted contribution features on the mobile web: WikiGrok and Collections

Services

Prototype revscoring service

Prototype other services to support mobile products (stretch)

Reporting pipeline

Finalize Unique Clients implementation requirements

Support the Dev team with the productization of Page Views reports

Continued responsibility for ad hoc traffic analysis and reports

Mobile *

Continue research support to WikiGrok and microcontributions

Research support for Project Gather (aka Collections)

Fundraising

Develop method for A/B testing with sampled data (stretch)

Develop proposal to run FR as a minimization problem

Formal collaborations

Wrap up first research project with Stanford (Improving linking structure)

Kick off second research project with Stanford (Increasing article coverage)

Process

Continue to provide ad hoc support to other teams, pending prioritization

Continue work with Team Practices on team process and iterations

Q4: Apr - Jun 2015

Goals dependent on review and discussion of Product team initiatives