In-context help and onboarding

Wikimedia Foundation projects In-context help and onboarding Group: Collaboration Start: 2018-07-01 End: 2019-01-22 Team members: Stephane Bisson, Benoît Evellin, Pau Giner, Moriel Schottlender, Elena Tonkovidova Management: Ryan Kaldari (engineering manager), Roan Kattouw (lead engineer), Joe Matazzoni (product manager)

“In-Context Help and Onboarding” is a project to improve retention of new wiki editors by giving them short tutorials and other training experiences on subjects that research shows impact new-user success. “In context” here means that the tutorials will be related to, and triggered by, users' current activities on the wikis. We might, for example, offer tutorials in response to user actions (e.g., clicking Edit for the first time), to events (e.g., a user’s first reversion) or at predefined milestones (the user’s nth edit). As a secondary benefit, we hope that this automated training system will make the work of volunteers who support new editors more manageable and reduce backlogs for patrollers.

The ultimate product—after appropriate small-scale experiments—might comprise a generalized Wikipedia Assistance service that communities can extend with training modules of their own devising. In-Context Help and Onboarding is a project of the Collaboration team.

This is an ongoing project; feel free to watchlist this page for updates.

Background

In 2017, the Wikimedia Foundation began a significant program of research under the title New Editor Experiences. The final project report opens with this problem statement:

Wikipedia requires a constant influx of productive new editors from diverse backgrounds to sustain itself and increase its quality and balance. However, for nearly a decade, the size of the project’s contributor community has stagnated.

The report goes on to document 11 “key findings” that analyze the top problems new editors face and recommend strategies for overcoming those obstacles. At the end of 2017, a cross-disciplinary team drawn from across the Foundation met in a series of workshops to explore ways to act on the report’s conclusions. After studying the 11 findings, the team selected four as the most important and actionable. To summarize those four prime findings:

• New editors’ greatest challenges are conceptual: they don’t understand wiki policies, including the “pillars” of Wikipedia like notability, neutrality and verifiability (finding #8).
• Among new editors’ conceptual gaps is that they fundamentally don’t understand how the wikis are made—by volunteers like them, through community processes (finding #5).
• In addition to the above conceptual issues, new editors struggle to master tools and editing processes. Meanwhile, the mechanisms that should make learning about those processes easier—notably talk pages and user help—are unintuitive and hard to discover (finding #9).
• The last prime finding recommends an approach: the best way to build editors’ skills is through iterative, progressive learning, rather than through large, one-time downloads of information (finding #7).

With these prime findings in hand, the workshop team next explored solutions. Of the many ideas discussed, In-Context Help and Onboarding was selected as one that best addresses the problems identified.

Principles & Approach

In-Context Help responds directly to the conceptual and informational gaps discussed above (findings #8, #9 and #5) The approach is progressive and iterative (finding #7). It augments the existing help-page system (finding #9). And the “push” nature of the interventions means this system has the potential to touch users broadly. In addition, the following key features and principles will guide the system’s design and ensure its success.

• Applied learning: Users will be more receptive to information that relates directly to their current activities and problems. By applying tests to the user’s contribution, (e.g., by measuring that X amount of text was added with no citation) we can target lessons more precisely.
• Effective communication. Written tutorials must be short and clear, distilling ideas to their essentials. Demonstrations and interactive training simulations are probably the best way to communicate processes.
• Coordinate the interventions. Logic will be required to ensure that users don’t get multiple help messages in response to a single action or see the same messages over and over. Help topics might also be sequenced or graduated, so that the same situation triggers a more advanced tutorial once the user has met certain conditions.
• Findable when needed. Some users may perceive just-in-time help as an interruption, to be dismissed summarily. As a backup, then, there should be a place where persistent New User Help modules can be accessed at a time of the user’s choosing (see mockup).
• Minimize intrusion. Because of the proactive nature of the interventions, we need to design so as to minimize frustration. For example, a specific “Newcomer Mode” preference, which a user might accept with the understanding that it entails extra guidance, could reduce resistance among more independent editors while freeing us to design more impactful experiences for those who opt in.
• Start small and iterate. Because we wanted to learn more about mid-sized wikis, the original New Editors Experience researchers focused on Korean and Czech Wikipedias. The In-Context Help and Onboarding system will likely be deployed and tested first on these two wikis, along with a few others yet to be determined.

Timeline

In-Context Help and Onboarding is currently in early planning stages. Research and design will ramp up in July, 2018, and active product development should start some time around January, 2019.

Feedback

We’re very interested to hear your thoughts on the project and approach outlined above. Where have you seen good examples of onboarding and automated help? What have you learned at your own wiki? Please add comments to the project talk page.

Right now, our focus is on thinking through the “content” that will best help new users succeed and the "triggers" that will ensure users get information in situations that will make it meaningful for them. See the draft list of what we’re calling “lessons and triggers” below. What does your experience tell you we can skip? What have we forgotten? We’d particularly be grateful for input from people who have experience supporting or teaching new wiki users. Please share your insights (in any language).

The sections below will be changed significantly in the coming weeks.

A draft list of ‘lessons’ and 'triggers' (updated 2018-03-22)

Below is a very preliminary list of possible training topics. What is missing? What can we skip? Remember, this system is designed to get new registrants past the first pitfalls in their user journeys; it’s not a comprehensive training course (though it can grow over time). What is the minimum set of interventions that, as a first release, can make a difference to new editor success?

• Library links: The “Library” links you’ll see below are to items in the Wikimedia Training Library. These are provided as examples of the type of content a given lesson might include; we won’t be using them verbatim.
• Groups 0-4: Think of these groupings as corresponding to experience levels. The lessons don't have a sequence, per se, but the same user action might trigger a Group 1 lesson for a very new user and a Group 2 lesson after the user has progressed.
• Triggers: The “lessons”  are meant to relate to, and be triggered by, users' current activities on the wikis. Getting the triggers right will be crucial to having users perceive our support as helpful and relevant. The examples here need more thought, so please share your ideas.

Group 0, Welcome

• Welcome sequence—segments include:
  • Wikipedia is created by users like you—be bold (cf Library).
  • Explain Newcomer preference—“You are now in Newcomer Mode,” and allow to opt out.
  • Connect to help resources: user talk page (where we'll have added newcomer help modules); explain special Newcomer help links they’ll see around, how to search for “help:subject,” direct to human help if available (e.g., Teahouse, Forum des nouveaux)?
  • If user didn’t include an email address, explain how having this can help us help them, and offer form to add now (cf Library).
  • End with a form: "Now that you’re here,  what do you want to do today?" 1) improve an existing article, 2) learn about how Wikipedia works, 3) communicate with a user 4) write an article.
    • Trigger: On registration
      • Actions that occur at the same time: 1) put a note on the user talk page from a local user welcoming and reinforcing offer of help and assistance (see example), 2) replace help icons on key tools/pages with Newcomer Help icons that link to Newcomer modules.

• Find an article to contribute to (suggest job queues for newcomer-appropriate jobs and/or subject-based projects—needs more thought).
  • Trigger 1: In the Welcome sequence, the user answers that they want to “improve an existing article.”
  • Questions: what are the most newbie friendly jobs?

• Find an article that needs to be written (not sure how this might work—needs more thought).
  • Trigger 1: In the Welcome sequence, the user answers that they want to “write an article.”
  • Questions: perhaps this should just be an early module in a revised Article Wizard tool?

Group 1, Things everyone should know

• “You’ve got a message”—introduce user talk page idea; introduce notification badges and panels (how you know you got a message).
  • Trigger 1: As above during the "Welcome sequence" we put a note on the use talk page. If the user sees the notification and goes to her talk page, launch lesson. But we don’t otherwise prompt or point to the message or email the user about it immediately after the welcome.
  • Trigger 2:  If the user doesn’t open the notification or talk page, send her an email after X days (one day?). This encourages the user to come back. The message contains a prominent link that launches the lesson.
  • Trigger 3: If user still hasn’t seen the message/talk page on next visit to the wiki, launch lesson.
  • Offer follow-on lessons: “Answer a user message,” “Contribute to a talk page discussion”

• Neutral point of view (cf Library)
  • Trigger 1: User makes contribution of a size large enough for POV to be an issue.
  • Trigger 2: As one of a small, perhaps rotating series of key modules offered as a verification step after first X edits, or until “permanent dismissal” (rough concept mockup).
  • Offer follow-on lessons: “The importance of including citations”

• Neutral point of view with warning that editing high-risk pages may lead to revert
  • Trigger 1: User makes an edit on a high-traffic or Featured article.
  • Trigger 2: User makes an edit on a controversial or protected page, identified as such by specified templates (E.g.,  Controversial topics, Pages under discretionary sanctions, Recent deaths, Semi-protected pages—and their cognates on each wiki.)
  • Offer follow-on lessons: “The importance of including citations”

• Verifiability (includes no original research) (cf Library and Library)
  • Trigger 1: As one of a small, perhaps rotating series of key modules offered as a verification step after first X edits, or until “permanent dismissal” (rough concept mockup).

• The importance of including citations ( and what are good sources) (cf Library, 2, 3)
  • Trigger 1: The user has made a contribution of X size without including a citation.
  • Trigger 2: As a (non-distracting) suggestion when pasting content of X size.
  • Trigger 3: As a follow-up suggestion to improve the page on Save when contributing to an article identified as lacking citations.
    • Note: requires a slightly different message from #1, which is triggered by user possibly making a “mistake.”
  • Trigger 4: The user contributes an article that gets tagged for no citations.
    • Note: since this will happen when the user is not on the wiki, we should 1) send the user a friendly, “Newcomer email” and notification about it, 2) pop a message next time the user returns logs in to the wiki.
  • Offer follow-on lessons: “How to add a citations”

• Notability (cf Library and Library)
  • Trigger 1: The user clicks on the “create” link after entering an unknown pagename or clicks on a redlink.
  • Trigger 2: the user saves an edit that includes no references to an article that carries a template questioning its notability.
    • (Note: this trigger would likely require a message with slightly different wording from that given in response to Trigger 1)
  • Offer follow-on lessons: "The importance of including citations," "How to add a citations,” “What is encyclopedic content?"

• Importance of  the edit summary (cf Library )
  • Trigger 1: The user saves an edit but does not marke the edit as minor or include an edit summary.
    • Note: this requires conflict management with lessons like Verifiability. E.g., the same action might trigger both, so Verifiability should get precedence, then this show up the next time.

• Assume good faith, be respectful, don’t panic if someone edits or remove your work. (cf Library)
  • Trigger 1: The one of the user’s edits gets reverted or undone.
  • Trigger 2: An article the user created is deleted or marked for deletion.
  • Trigger 3: Upsell from “Contribute to a talk page”
    • Note: since #1 and #2  will happen when the user is off-wiki, we should 1) send the user a friendly, “Newcomer email” and notification about it, 2) pop a message next time the user logs in.

• How to make a citation, Visual Editor/citoid (cf Library, Library, tutorial )
  • Trigger 1: An editor using Visual Editor clicks the “Cite” button for the first time.
  • Trigger 2: The user clicks a new “New User Help” icon/link we add to the Citation tool.
  • Trigger 3: As a  follow-on lesson from “The importance of including citations”.

• Answer a user message (indenting and signing) (cf Library, Tutorial, video)
  • Trigger 1: The user gets a message on her user talk page (excluding the initial Welcome message).
  • Trigger 2: The user is mentioned in a discussion or edit summary.
  • Trigger 3: As an upsell from the “You’ve got messages” and “Assume good faith” lessons.
  • Offer follow-on lesson: “Assume good faith, be respectful”

Group 2, Basic skills and more concepts—very preliminary ideas

• What is encyclopedic content—Wikipedia is not a dictionary, directory, collection of documents…. (cf Library)
• Join in a talk page discussion (purpose of talk pages, basic Wikitext and etiquette) (cf Library, Tutorial)

• Content is free and must be copyright free (cf Library and Library)
• You can watch a page (Watchlist) (cf Library)

• Conflict of interest
• Intro to editing—basic VE overview, visual vs. source editing, switching between the two,   (cf Library  and Library)

Group 3, Going deeper—very preliminary ideas

• Write an article...
• Explain about template messages
• Biographies of living persons
• Something about harassment/Friendly Spaces policy (cf Library)