Growth/Discussing potential Growth ideas

Summary

Update 2018-09-20

Over the last month, the Growth team received a huge amount of valuable reactions and comments from many different communities on this page's discussion page, in Czech and Korean Wikipedias, and through other communication channels. Below, we've summarized what we heard about each of the eight ideas, and we'll be asking some follow-up questions on the discussion page. This information is making a big difference in our team's planning. To continue to follow along and have input on this project going forward, please watch our project page and sign up for our team's newsletter.

Original summary

The WMF Growth team is beginning the process to determine which new development work to experiment with in Czech and Korean Wikipedias. Building on the New Editor Experiences research in which those two communities participated, the objective of the work will be to increase the retention of new editors. We now have a list of ideas that come from the research findings, and we want to hear everyone's thoughts and reactions. Please use the talk page to discuss! This information will also be posted and discussed on Czech and Korean Wikipedias, in those languages.

We're hoping to hear the majority of the communities' thoughts within the next two weeks (by 2018-09-05).

In Czech and Korean Wikipedias, between 1,000 and 2,000 new accounts are created each month (see Czech data (Archived 2020-01-24 at the Wayback Machine) and Korean data (Archived 2020-01-24 at the Wayback Machine)), but only a small percentage of those users make edits and continue to edit after their first day. Over the next few months, our focus will be to improve the initial experience of new logged-in editors so that they learn what they need to continue to edit.

Instead of committing to one large project, the Growth team plans to begin with small projects and learn. We're now ready to determine which small projects to begin with, and it is critical that communities tell us about ideas that they think will and will not be impactful. To get the conversation started, we have listed some ideas that seem to fit well with the results of the New Editor Experiences research. That research showed two types of features that could make an impact:

Human-to-human help: features that connect editors to each other so that experienced editors can help newer editors succeed.
In-context help: changes to the Wikipedia editing experience to show guidance around technical and conceptual challenges at the time it is needed.

Our list of potential ideas below are some of our team's favorites that fit into "human-to-human" or "in-context help". They are just the beginnings of ideas, and we know that there are a lot of details, background, and complications that we would need to figure out. We first want to talk about whether the concept is good. We also know that these ideas may not make sense in many wikis, and that's why we're posting here. We would like community members to respond with:

Thoughts or reactions to the ideas on this list. Could they work well? Would they be problematic? Have similar things been attempted?
Other ideas that could help retain new editors, whether or not they are on this list.

This is not a vote to decide on ideas from the list -- it's a discussion!

As a next step, we'll take everything we learn into account and narrow down to a couple ideas that seem to make the most sense to work on first. Then we'll work closely with communities to design and implement initial versions. Our team plans to build small things and learn quickly -- we won't be afraid to change our plans when we learn.

To sign up for the Growth team’s newsletter, please add your username here.

Potential ideas

In-context questions or chat (1)

Comments about that idea

Problem: When new editors are working on a page, it is difficult for them to find a place to ask a question. They have to leave the page to go to the Help Desk, IRC, or some other talk page.
Potential solution: Make it possible for an editor to type a question in the editing environment, without having to go to another page.; At first, this might be a menu where they submit a question that gets posted to a wiki’s existing Help Desk.; This capability would be more prominent for users who we know are new editors.; If this works well, we might talk about evolving this into realtime chat, which would be a one-on-one in-browser chat between a new editor and an experienced editor.
Issues: Something similar already exists in Visual Editor, in which users can report bugs, and editors frequently use it to ask for help.; There are also similarities to the previous MoodBar project.; We would have to be sure that there are enough experienced editors to answer questions.; We might also accompany this with work that notifies a user via email when their question is answered.; We would need to decide where to post the questions. On the one hand, technical questions might belong in Help Desk, whereas questions about the reliability of a source might be relevant for the WikiProject related to the article.
Post-discussion summary: In general, it sounds like this is a great idea that would have many implementation challenges and questions. On the positive side, this idea would allow new editors to get help immediately, and would put them in touch with human editors who can answer many different types of questions. Chat is also a familiar way of asking questions from other modern websites. On the negative side, the biggest question is capacity: whether there are enough experienced editors to quickly answer questions coming in, especially if implemented as chat. There are also questions around how chat interactions would be kept safe and potentially made public. Other considerations include whether we can learn from community experience with MoodBar (a project with a similar feel), and from IRC channels dedicated to answering editing questions.

In-context lessons (2)

Comments about that idea

Problem: New editors frequently are confused on how to edit, but the materials that teach them how are not near the editing interface.
Potential solution: Add help content to the actual editing experience for new users so that they can learn about things like citations, wikitext, and adding links, all without leaving the editing interface.; This might take the form of pop-up boxes that can be dismissed.; For instance, when a user clicks "Insert -> Template" in the Visual Editor, a box might describe what templates are and when to use them.
Issues: It's possible to start with only one or two of these and test their impact before building many of them.; The visual editor already has a few existing dialogs that are similar to this idea. Their impact is currently unknown.
Post-discussion summary: This idea received a huge amount of positive feedback, including from actual new editors. It's clear that new editors need guidance as they make their first edits, and teaching content embedded in the editing experience can scale to many new editors without adding to the workload of experienced editors. There was also discussion of the value of videos to be these in-context lessons. On the negative side, though an automated lesson scales well, it is not as flexible as a human, which can answer hundreds of different questions. It sounds like if we implement this idea, we should be careful about adding too much cluttering content to the editing experience, and also about how to make it possible to view the content while also editing. We would also need to consider what, if anything, should be implemented for source editing (not just Visual Editor). One piece of feedback we heard from a new editor is that it would be helpful to have warnings before publishing that the editor is about to make a mistake, such as not including citations. Some editors pointed out that a couple small in-context lessons exist in Visual Editor already, and that it might be possible to look at data on how new editors interact with them.

New editors suggest edits (3)

Comments about that idea

Problem: Research shows that some new editors are afraid to be bold and publish their edits because they are afraid to do something wrong. We also know that many new editors are reverted, which causes them to stop editing.
Potential solution: Make it possible for a new editor to voluntarily "suggest" an edit instead of "publish" it. This would be an option, not a requirement, for a new editor.; In the near term, this might be a workflow in which a new editor posts the suggestion to the project’s talk page.; In the longer term, this might be a structured workflow in which new editors can write their edit and suggest it, and experienced editors can merge it in themselves.; This might make it more likely for new editors to be engaged in discussion about their edit, as opposed to being reverted.
Issues: There is an existing system for approving edits made by new editors in German Wikipedia before they are published.; There is an existing system for suggesting edits on protected pages used in English Wikipedia.; We would need to be mindful of edit conflicts.; May attract vandalism or low quality suggestions and cause more work for reviewers.; We would have to think about how much additional work this creates for experienced editors.
Post-discussion summary: It seems that this idea might be the one with the fewest positive opinions, and biggest open questions. Similar systems are in use on German Wikipedia, Arabic Wikipedia, English Wikibooks, and other places. Those systems are working well there, but it is unclear what impact they have on new editor retention. While making it easy/possible for new editors to suggest edits might make their initial editing experience more accessible and less intimidating, it could also make things more complicated and confusing. Such an implementation would also add another queue that experienced editors would have to review. Before going down this path, the team would need to do a good deal more research and design thinking.

Focus on help desk (4)

Comments about that idea

Problem: Most wikis have a help desk where new editors can ask questions, but most new editors don't know about or find the help desk.
Potential solution: Invite new editors to the help desk by posting on their talk pages, using banners, or using email.; This might come along with other improvements to the help desk, such as listing new questions at the top of the page or notifying new editors when their questions have been answered.
Issues: The most successful process for retaining new editors in English Wikipedia is the Teahouse, a friendly help desk to which new editors get invited. It has a 10% increase in new editor retention.; By driving more traffic to the help desk, we'll need to consider whether there are enough experienced editors to answer questions.; Many wikis already have bots that post welcome messages on new editor talk pages, which includes a link to a help desk. This work would make the invitation much more explicit.
Post-discussion summary: This idea has a lot of positive opinions. Many wikis already have help desks with experienced editors productively answering questions, and there exists proof that help desks can increase retention. The main work here would be to drive new editors to those help desks. One potential issue to watch out for is whether driving too much traffic to help desks will overwhelm the experienced editors who answer questions, and also give new editors a bad experience while they wait a long time for answers. In addition to inviting new editors to help desks with a bot, the idea was brought up of making a link to the help desk a prominent part of a new editor's experience while they actually edit. This implementation would have the potential of helping us learn whether the "In-context questions and chat" idea from above is viable. There was also much discussion around how to improve the new editor's experience of the help desk so that it is easier to ask questions and receive replies.

Email new editors their impact (5)

Comments about that idea

Problem: Research shows that new editors can be inspired to continue editing when they see that their work is impactful, but they don't know how much traffic their content is getting.
Potential solution: Email new editors with the numbers on how many page views their articles are getting, or the articles that they have edited.; Similarly, we could email new editors to encourage them to return to the wiki and continue editing, or to recommend next tasks to do given what we know about their work already.; Through email, we can reach lapsed editors who won't see notifications on wiki.
Issues: Not all users sign up with email addresses.; We will need to be careful not to email users too much.
Post-discussion summary: Many experienced editors who spend time with newcomers found this to be a good idea because of how newcomers have been motivated by finding out that people read their work. There have also been good experiences with the Outreach dashboard, which shows the number of pageviews to articles created in specific events. On the other hand, there are some concerns that if more communication is done through email, new editors will have a harder time learning to communicate on-wiki. This is something we would have to think about. We would also have to think about the logic that causes the emails to be sent and populates them with the information that would be motivating. For instance, what to send to a new editor who edits 10 different articles in their first week? Or who has received zero pageviews on the articles they have created? We will also need to look at what percentage of editors sign up and verify their email address, as this number will dictate how effective such an idea can be.

Personalized first day (6)

Comments about that idea

Problem: When new editors create accounts, they are simply redirected back to the page where they were before. They do not get any additional guidance or content, and nothing suited to their individual interests or needs. This is a missed opportunity to guide new editors at the moment when they need it most.
Potential solution: Ask new editors additional questions during their account creation process, such as their reason for creating an account, what they are trying accomplish, topics that they are interested in, or whether they want to meet a mentor.; After the login is created, we can direct the new editors to help content that is relevant for their goals, or to WikiProjects that match their interests, or potentially match them with a mentor who shares their interests.
Issues: Additional questions would need to be optional.; We would want to make sure that this doesn't make registration too difficult.; We can add these questions before we decide exactly how the new editor experience will change according to the answers. The learning from the questions themselves will be valuable enough.; There is currently an experiment underway for matching new users with other editors based on questions asked near to their registration.
Post-discussion summary: This idea has a high potential for learning, but it's important that additional questions do not deter new editors from creating their accounts. As a baseline, we will want to find out what percentage of people who start the signup process do not complete it. It was pointed out that some wikis have Guided Tours implemented that show new editors a little about editing and navigating right after they create accounts. We would need to be mindful of which wikis have those so that we do not get in the way of those tours. Editors from a few communities suggested that right after account creation could be a good opportunity to let new editors know about local events, since off-wiki events are productive for retention and engagement.

Understanding first day (7)

Comments about that idea

Problem: Most new users who create accounts do not ever make edits. We do not have any knowledge of what they do during the time they are logged in -- whether they read help content, attempt edits that they do not publish, or something else. Knowing more about these initial sessions could help us make the experience better.
Potential solution: Implement instrumentation to the software so that we gather data about what new editors do shortly after creating their accounts. (The team would be careful to apply appropriate privacy policies here to protect users, which can include aggregating, anonymizing, and purging the data.)
Issues: This would not change the experience of editors at all; it would only be gathering data for research purposes.; We would be careful to respect user privacy and our data policies.
Post-discussion summary: It seems that there is unanimous agreement that this information would be valuable to know, and also unanimous importance placed on being careful with privacy around this data. We'll also need to look into the instrumentation that already exists on Visual Editor usage, to see what we can use for our team's purposes.

Email notification replies (8)

Comments about that idea

Problem: We suspect that many new editors receive email notifications about posts on their articles or talk pages, and that they simply reply to those emails. When replying to an email notification, the reply goes nowhere and therefore never gets answered.
Potential solution: Route these email replies to OTRS so that new editors can stay engaged and learn how to communicate on-wiki.
Issues: We are not sure how often this occurs. Some initial research is needed.; We would need to route replies to the appropriate language OTRS.; We would need to make it clear to users where their email will go and who will read it.
Post-discussion summary: The most important takeaway from this discussion is before deciding to work on this problem, we need to find out how often users actually reply to notification emails. There was much conversation of whether routing those replies to OTRS is a good idea. On the one hand, the OTRS queue could give productive responses, but on the other hand, the volume of email could further burden OTRS queues that are already too long. An exception might be that smaller wikis do have OTRS operations that could handle additional traffic. Setting OTRS aside, several editors then brought up the idea that it could be more scalable and productive to simply improve notification emails by making them easier to understand, and making it clear how to answer them on-wiki.