New Engagement Experiments
|
New Engagement Experiments
Exploring experiments to increase the engagement of existing audiences of readers/users and contributors on Wikipedia, to ensure we continue to grow Wikipedia as a destination for our current audiences.
|
Background
[edit]This work is one of the KR results in the FY24-25 Annual plan Objective, Product & Engineering Support 1 (PES1): Make the Foundation's work faster, cheaper, and more impactful.
Staff do a lot in their regular work to make our operations faster, cheaper, and more impactful. This objective highlights specific initiatives that will both: a) make substantial gains toward faster, cheaper, or more impactful; and b) take coordinated effort and change of formal and informal practices at the Foundation. Essentially, the KRs included in this objective are the hardest and best improvements we can make this year to operational efficiency of work touching our products and technology.
PES1.3 New Engagement Experiments Key Result
[edit]Run and conclude two experiments from existing exploratory products/features that provides us with data/insights into how we grow Wikipedia as a knowledge destination for our current consumer and volunteer audiences in Q1 and Q2. Complete and share learnings and recommendations for potential adoption for future OKR work in the Wiki Experiences bucket by the end of Q3.
Context
[edit]This work is a counterpart to the Future Audiences objective, but focuses instead on uncovering opportunities to increase and deepen engagement of our existing audiences (of Wikipedia consumers and contributors) through more nimbly testing more on-platform product ideas.
It lives in PES1 as it is an energiser and multiplier - channelling the time individuals and teams have already devoted to hacking/experimenting on side projects to bring more promising features into focus. Instead of these side projects languishing (not a good use of our limited resources), this KR provides a path for some of these ideas to potentially make it into larger APP setting through proven experiments, thus more efficiently using staff time and motivating their creativity and productivity.
This will make our work more impactful and faster as it helps the foundation to align on the correct goal in less time. By shepherding more of these smaller, shorter projects into play, we also diversify our spread of ‘bets’ for more learnings and trials of ideas that may transform Wikipedia in line with the changing needs and expectations of our current audiences.
Metrics & Indicators
[edit]- Milestones: Two experiments completed* about making Wikipedia grow as a destination by the end of Q2, with recommendations* to inform Wiki Experiences planning.
- Second-level / Experiment-specific success metrics: Second-level metrics will be related to the specific end-goal of a project (eg. engagement measures like sharing, revisiting, time spent)
Hypotheses/Projects
[edit]The three hypotheses below are the specific things we are doing to address the PES1.3 key result.
Hypothesis: If we provide insights to audiences about their community and their use of Wikipedia over a year, it will stimulate greater connection with Wikipedia – encouraging greater engagement in the form of social sharing, time spent interacting on Wikipedia, or donation.
- Primary success metric - One completed experimental project that provides at least one recommendation about “Wikipedia insights” as an opportunity to increase onwiki engagement.
- Secondary metrics - Measures to determine success of the experiment itself in increasing engagement. Potentially:
- Interactions - Views, Time on Site, etc
- App installs
- Donations
- Sign-ups
Status: In-progress
Hypothesis: If we create Wikipedia-based games for daily use that highlights the connections across vast areas of knowledge, it will encourage consumers to visit Wikipedia regularly and facilitate active learning, leading to longer increased interaction with content on Wikipedia.
- Primary success metric -At least one completed experimental project that provides at least one recommendation about gamification of learning as an opportunity to increase onwiki engagement.
- Secondary metrics - Measures to determine success of the experiment itself in increasing engagement. Potentially:
- Interactions - Views, Time on Site, etc
- Account creations
- App installs
- Donations
- Proposed games
- Wikipedia Linking game (eg. Wiki Racing, the Wikipedia game, speedrun) - The ‘classic’ Wikipedia game, where players are given two articles and must find the shortest or fastest distance between them using blue links.
- Wikipedia Trivia game - A daily trivia game utilizing Wikipedia content to create trivia questions.
Status: In-progress
Hypothesis: If we develop a new process/track at a Wikimedia hack event to incubate future experiments, it will increase the impact and value of such events in becoming a pipeline for future annual plan projects, whilst fostering more connection and enthusiasm from engineering/design staff and/or volunteers to experiment and engage more with strategic prospects.
- Primary success metric - One PES1.3 project is initiated and/or advanced to an OKR from this event.
Status: Completed. Pending report of recommendations.
Experimentation completion criteria/definition
[edit]Criteria for an experiment being “completed”:
- An experiment must receive internal validation (e.g., “does it work?”, as measured by internal quality assessment/beta testing, concept and/or usability testing). An experiment may be undertaken and fail internal validation – this is fine as long as insights into why and next steps are documented. (focus of the PES1.3.3 hypothesis)
- If successfully internally validated, an experiment must then be externally validated (e.g., “do people want it?” as measured by real-world usage metrics and qualitative user feedback)
- Key insights from both internal and external validation (the latter if applicable) must be documented and shared internally (at WMF staff meetings) and externally (via the mediawiki/metawiki pages, Diff posts).
Criteria for at least one “recommendation”:
- Recommendations from should appear in objective or KR text or context in the WikiExperiences Product & Tech bucket, and/or other department’s OKRs during annual plan OKR drafting in Q3.
- Recommendations may take the form of stating what is intentionally not being pursued in light of the experiment findings.
Experimentation vs. WE work
[edit]PES 1.3 work is experimental at its core, it is not intended to be WE roadmap work.
The goal of this work is to be lightweight and efficient by creating small, time-limited experiments utilizing limited resources. The focus is to have a limited impact on product teams and limited dependencies. While this work can inform future WE roadmap work, the work produced under PES 1.3 hypotheses should be built in such a way that it can be easily ‘sunset,’ at the end of their time-limited experimental runs. The work on PES should therefore be reflected as such in the teams’ roadmap.
Note: the emphasis is on informing roadmap of On wiki experiences, the experiments themselves should not need to be done on production.
Team
[edit]Part of the intention of the this work being in the PES1 objective is about channelling efforts that individuals and teams already devote to hacking/experimenting on side projects in a way that brings more promising features into focus for development in future Annual Plan initiatives. The core team is focused on project planning and resourcing coordination for the various experiments.
