Talk:Growth/Positive reinforcement/2023

I love Special:Impact, but can it be set to 20 instead of 5 articles?

You'd like to have it being configurable?

Absolutely! I'd love to see a much longer list; not just because the article I thought would be at the top did not appear.

Support for this proposal. Also I didn't know about this page, maybe once it has more features it could be linked at the bottom of one's Contributions page. I think it doesn't work as one thinks it does: it only checks x recent edits or so, not all pages edited within the last 60 days.

It could be very useful and have a large impact on editor retainment and for newcomers as a motivation. In normal cases, pageviews are nearly all the feedback editors receive (assuming their edits are constructive). I think the reason for why it can't show more is probably server load, but maybe this problem could be mitigated somehow, for example by not requiring server queries to get these pageview numbers but storing the pageview counts with editor associations right away. That's probably why the following isn't yet implemented:

It would also be nice if you could easily see in a sortable table how many times your uploaded images have been viewed (direct views + pageviews of all the Wikimedia pages that use them). Maybe I should create a phabricator issue for that albeit due to server load it probably would first need changes to how pageviews data is stored.

A link to Special:Impact can be locally added by the communities wherever they want.

This impact page was designed to show to newcomers how many reader benefit from their edits. The end goal is to indeed motivate newcomers and retain them. This page is shown at Special:Homepage (here at en.wp), along with other tools to motivate newcomers to participate.

However, this page has limitations. The data on this page is only updates once a day (to avoid huge calls on the API) and the data shown is based on previous page views, not page views since you made an edit to the page.

The Growth team built it as a special page because it was technically easier, but it wasn't designed to be used by experienced users. Maybe it is not a fit for them.

We work on easing newcomers' first steps, hence we are not considering improvements designated towards experienced users, but we welcome suggestions regarding improvements for later iteration on this module. I documented the possibility of personalizing the number of articles shown; feel free to create Phabricator tickets with your other ideas!

You probably ended here because you saw the info in Tech/News: we are working on a new, extended version of the Impact module. You can discover it here (assuming that you both come from English Wikipedia).

So it doesn't show the pageviews since your edit to the page like "Most viewed (since your edit)" suggests. However, I also edited a page within the last 60 days which got more pageviews than the 5 shown there. From Sandizer's 2nd comment it seems like he has the same issue and so far it seems like a bug. Thanks for creating that phabricator issue.

New Mentor Dashboard module:

As part of the Growth team's Positive Reinforcement project, we plan to add a new module to the Mentor Dashboard.

This module will surface mentees that match certain criteria. For instance, in the screenshot, you will notice a mentee who made 20 edits total and has no reverts over the last 48 hours. Some of these parameters can be changed in the module’s settings. We also display some other user metrics like "Thanks received" and "Longest streak" of days edited in a row.

Then, Mentors are invited to “Send appreciation” to this new editor (possibly after reviewing their edits or talk page). These messages of appreciation may be prefilled with a default message (which can be customized in settings). You can edit this message before posting, as we will redirect you to the newcomer’s talk page to post the message.

Multiple studies have found that newcomers’ editing activity is increased when they receive thanks and other positive messages about their work from an experienced editor or mentor [1].  We designed this module with the hope that it will improve new editor retention with a minimal time-investment from Mentors.

Our questions to you about this design are:

• Would this new module encourage you to send appreciation to your mentees?
• If you had to provide appreciation to this mentee, do you have all the information you need to decide to praise this user?
• Do you think some Mentors will hesitate to “Send appreciation” because it’s not clear that there is an additional step (visit the talk page and confirm message) after clicking that button?
• Do you have any comments or suggestions regarding this feature?

Settings:

We want this new module to work for everyone, so we plan to make the settings configurable by Mentors.

First, you can define the number of edits within a given time frame a newcomer needs to make to be suggested as praiseworthy.

Then you can define the default message that will be used when you send praise to the newcomer. Both the subject and the content of the message can be pre-defined.

Finally, you can decide to get a notification when a mentee matches your selection settings.

Our questions to you:

• Is the edit time frame setting enough to cover your needs to find mentees to praise?
• Should we provide a pre-defined message you can edit later?
• Would you use notifications to be informed of a new mentee to be praised?
• Is there a setting you’d like to see in the module’s settings?

Thanks in advance to anyone willing to provide feedback!

Note: we are also currently discussing this on English Wikipedia [1] and on Arabic Wikipedia, Bengali Wikipedia, Czech Wikipedia, and Spanish Wikipedia.

Hello @HLHJ @Mike Peel @Czar @John Cummings @T Cells @Pigsonthewing @Sdkb @John Broughton @Pelagic @NickK @LittlePuppers @Zoozaz1 @Nick Moyes @Galendalia @Barkeep49 @NGC 54 @ValeJappo @Ата --

I wanted to invite you all to a discussion on the feature set the Growth team is working on next. This "positive reinforcement" effort is our next big project following "add an image", which many of you helped guide. The idea behind "positive reinforcement" is that newcomer tasks (like "add a link" and "add an image") are doing a good job of helping newcomers make their first edits -- but they aren't having a clear impact on getting newcomers to come back another time to make more edits. In other words, they are helping with "activation", but not with "retention". We think the "positive reinforcement" project could increase retention by showing newcomers the impact they are having and encouraging them to continue. Also, when we discussed newcomer tasks with communities, many of you wanted to make sure that newcomers wouldn't get stuck doing easy tasks, and that they would progress to more valuable kinds of tasks. With positive reinforcement, we can encourage that progression. But we know that the wikis are not like other apps and website -- we don't want to motivate people to make bad edits just to rack up points and awards. So we definitely need community input to figure out how to motivate newcomers in the right way that fits our communities.

I also want to tell you about an important transition and introduce you to @KStoller-WMF. I'm moving into a new role at WMF as the manager over the Growth, Editing, and Web teams. Kirsten will be the new product manager for the Growth team, and we're excited about her background as a librarian and as a product manager for a website that also has an active community. Going forward, you'll mostly be interacting with Kirsten around Growth product work, but I will still be working closely with her on strategy and tying projects together across multiple teams. Thank you all for your help building the Growth team features over the previous years, and Kirsten and I are looking forward to continuing to learn from you as we keep making the wikis a better place for newcomers!

Back to positive reinforcement: it would be great if you can check out the background and some of the design ideas on the project page, and post any of your thoughts, concerns, and questions in this thread or in a new one. Here are some prompts to think about:

• What has worked well on your wiki for motivating newcomers?
• Which of the ideas on the project page do you think have the most promise? The least promise?
• What could go wrong and what should we try to avoid?

I'm fairly certain that the WMF tried, maybe five or so years ago, thanking new contributors (on their User Talk page, I think). I believe this was done randomly, and I think the conclusion was (a bit more nuanced that this, but basically) that it made no difference. So, if my memory is right [sorry, unable to find a page to point to, regarding that effort/project], personal praise isn't a magic bullet.

Thanks for the feedback @John Broughton! I’m not sure which study you are thinking of, but you are right, personalized praise definitely won’t be a magic bullet. But based on several past studies we think it’s reasonable to expect this idea to be a step in the right direction.  

Here are a few of studies we looked at that found that newcomers’ editing activity is increased (at least in the short term) when they receive thanks and other positive messages about their work from an experienced editor or mentor.  Here are few of the studies we looked at:

• WMF Research:Understanding_thanks  
• Citizen and Tech study: Effects of Saying Thanks on Wikipedia
• Zhu, Haiyi & Zhang, Amy & He, Jiping & Kraut, Robert & Kittur, Aniket. (2013). Effects of peer feedback on contribution: A field experiment in Wikipedia.

I think one of the major challenges this idea faces is that reviewing and thanking newcomers requires time from contributors. We are still brainstorming ways to surface praise-worthy edits to mentors or other contributors. Do you have any feedback? How do you think we should flag contributions that might be worthy of thanks?

Hello Kirsten - always good to have more library folk on board! And good luck in your new role, Marshall.

We show editors a "your edit has been saved" notice each time that happens. Perhaps for new accounts, we could mae use of that? Something like "Your edit has been saved - you're doing great work - why not do another one"?

Thanks for the warm welcome, @Pigsonthewing!  

Nice, it sounds like we are thinking along the same lines as you.  Here’s a mockup of one idea we are considering, which displays an encouraging prompt after a newcomer edit is saved. Is this similar to what you are thinking? Do you have any ideas for improving this idea?

Generally yes, but I'm not so keen on the "1/5", "5/5", "daily goal" part. Is that goal set by the user? We shouldn't be discouraging people who might otherwise make more than five edits, nor encouraging people to make unnecessary edits just to hit an arbitrary target.

Great, thanks for the feedback! I also have some of the same hesitations regarding the daily goal. I assume allowing users to set and customize their own goals would likely be more motivating. Customization would also add to the complexity of implementation, but it's certainly worth considering. We'll keep this in mind as we take another look at the designs.

As the project page says, "We believe that these reinforcements from real people would be more effective than automated ones coming from the system."

A general suggestion: I think it's worth learning from other sites how they do it. I think that often involves, e.g., an email a day later saying thank you for your first contribution, have you considered coming back to do more, and here's some suggestions? This is perhaps more what newbies will expect, and less what experienced Wikimedians would want? But I'd also suggest contacting people that work actively on this (I don't), I know @WereSpielChequers has worked on this, and there's lots of on-wiki projects that would have relevant expertise (new page patrol, Teahouse, etc.) - and always good to ask new editors who're just past the first edit stage, what kept them going?

Thanks Mike, I spent a lot of time on Quora in recent years, and there there was much more automated feedback on things like the number of people who'd read what you'd written. Obviously not worth mentioning if figures are low, but good to send people feedback if what they've done is read by a lot of people, we do this with the DYK process but could do it more widely. As far as Wikipedia is concerned, I think there is a big potential in tailored invites - people's first edits usually indicate some of their interests, and we could auto generate invites from relevant WikiProjects. Even if the WikiProject concerned is not currently active, it is likely to contain a bunch of resources and list some relevant and potentially tempting tasks.

Thanks, @Mike Peel and @WereSpielChequers. Agreed, it's fairly standard to receive a series of welcome emails when joining a new site. We've recently started some work in that direction, and plan to closely measure the impact: Engagement emails

One of the Positive Reinforcement ideas we are discussing includes Personalized Praise which could result in an email depending on the implementation method we pursue (along with user preferences and if the new user has an email address associated with their account). I think the main challenge we will face with that project is finding ways to encourage mentors and other contributors to spend time providing praise to newcomers who make high-quality initial edits. How do you think we should flag contributions from newcomers that might be worthy of thanks?

I think there is a big potential in tailored invites - people's first edits usually indicate some of their interests, and we could auto generate invites from relevant WikiProjects.

Currently the Growth team is using the Welcome survey to provide some personalization on the newcomer homepage. But certainly there is more we could do here. I love the idea of connecting users to the topics and WikiProjects that would appeal to them. Newcomers have so much to learn and read when they get started, it's easy for them to get lost and never find the right project or task that appeals to them. I wonder if we could use the topics a newcomer selects as interests as a way to also suggest a WikiProject that relates to that topic. Is that what you were thinking?

Do you have any other feedback or concerns about the Positive Reinforcement ideas and designs?

Hi KStoller, I was thinking in terms of the Wikiprojects tagged on talkpages of articles newbies have edited or that are relevant to categories those articles are in. This has the drawback that some tags will be completely irrelevant to the user, however transparancy could be maintained if you send hem an invite that contains links to all the relevant Wikiprojects - they can then hone in on the ones they think are relevant to them. Hi <insert name hare> based on your first ten edits, we thought you might be interested in some of the following Wikiprojects Vulcanology, mountaineering, Washington state and skiing - WikiProjects are groups of Wikipedians like you who have an interest in a certain area. OK they may have edited that article on a mountain because of the botany or abandoned radar base, but they will have picked up the principle that Wikiprojects exist and there might be ones of interest to them. As for encouraging praise of newbies, I'd add some software to the welcoming process, enable those who welcome newbies to get feedback on those newbies such as their 100th or 500th edit, with an easy opt out such as these are newbies who you welcomed and whose userpage you still watch. You could also create a page on WikiProjects that enables members to give thanks to new editors who have begun to be active in that area, though I suspect that has been covered by watchlisting etc. Perhaps a better route would be to send messages to active Wikipedians who haven't yet issued thanks or barnstars explaining how to do so now that you've done your thousandth edit, have you considered encouraging Wikipedians who you see doing good edits, either by using the thank button or giving them a barnstar? WereSpielChequers (talk) 09:07, 19 May 2022 (UTC)

Thank you for detailed feedback, @WereSpielChequers.

I agree, there is more we could do to personalize suggestions based on what a newcomer has edited. The newcomer homepage currently suggests edits based on topics the individual has selected as interests, but certainly there is more personalization we should consider as we further onboard newcomers.

And thanks for all of the ideas related to encouraging personalized praise of newcomers. As you've noted, there are definitely many different routes we could consider. Many of the ideas we have discussed involve mentors, but I like that your ideas aren't limited to just that small (and likely already very busy) subset of contributors.

Thanks KStoller, this business of newbies selecting topics of interest is new to me. Unless we are talking about userpages, either it didn't exist in 2007 or I've forgotten about it. Can you tell me what proportion of new editors indicate such interests? If it is the great majority then this might be a more effective route than the one I detailed. WereSpielChequers (talk) 10:32, 20 May 2022 (UTC)

Hi @WereSpielChequers. This is a relatively new feature, so not something you would have encountered in 2007.

As part of the newcomer Homepage and newcomer tasks, we allow newcomers to select interests from about 40 topics. Here's an example screenshot. We then use that to personalize the articles / edits we suggest.

I'll have to dig into metrics to see how many newcomers complete this step of the onboarding funnel and select interests. Parsing the data is slightly complex since the percentage of new accounts which have the newcomer homepage and newcomer tasks enabled by default varies by wiki. But certainly for users who have self-selected interests, it's likely to be a fairly accurate way to understand which topics or WikiProjects they might find appealing.

If you are interested in checking out the newcomer Homepage, here's info in enabling the Homepage. And if you have any feedback, please let me know. Thanks again for the feedback about the Positive Reinforcement project!

Thanks, that's interesting. I see a disconnect in that forty topics is too few to match with the Wikiprojects. So Sports and Biography are in the forty, but Wikiprojects function more at the level of professional wrestling or Baseball. Sports is an umbrella. WereSpielChequers (talk) 00:14, 21 May 2022 (UTC)

You are totally right, the topics are very general, and the Wikiprojects can get very specific. It's also worth considering that the smaller and medium wikis will have fewer Wikiprojects than English. And I believe that some small wikis have no Wikiprojects, so it's perhaps not an idea that would work for all wikis.

But I still love the underlying idea: that as newcomers start advancing and trying more difficult tasks, eventually we should expose them to more advanced ways to contribute and involve themselves in the community. Learning to contribute as a newcomer feels like a labyrinth of technical, conceptual, and cultural challenges, and there's so much more we can do to help ease newcomers along the path to becoming successful contributors.

Sorry, yes I was thinking of English. Taking Georgian as an example, it is a few years since I went to a Tbilisi meetup, but they had a monthly focus project that a lot of them would participate on. If that's still their modus operandi, for that wiki you'd likely promote the idea of a monthly topic. What seems obvious to me, is that this requires asking the various wikis what process we should promote to newbies. Currently because of our software limitations we don't have autosign on talkpages, so we have to promote the idea of signing with four tildas in all welcomes to Newbies. Fix that problem and we have a lot of communication bandwidth that could be repurposed for a message more positive than "Our software is so klutzy you have to use this symbol you've never heard of to sign your posts".

Thanks for the feedback and example, @WereSpielChequers. Your Georgian example is in line with a concept the Growth team wants to focus on more: ensuring Growth features are extensible and customizable to better fit the unique needs of each wiki.

And you are right, manually signing talk pages is definitely not intuitive to newcomers. I know the Editing team has many Talk Page improvements in the works that should benefit newcomers. I really hope we see an improvement in new editor retention as we start to remove some of these common stumbling blocks and the Growth team works on features to ensure newcomers receive positive reinforcement along the way.

Thanks again for taking the time to chat through these ideas!

The Growth team completed user testing of Positive Reinforcement designs and wrapped up some other projects, and we are finally more focused on moving the Positive Reinforcement work forward! In an effort to share early and often, I wanted to post an update:

Check out the initial improvements to the Impact module in Beta. The Impact module (the "Your impact" section) is still incomplete and missing the article list, but you can see some of the initial work on adding in more personalized info and data visualization.

If anyone has any initial feedback, I would love to hear it! I'm especially interested in:

Should the Impact module's "Thanks received" count link the Thanks log? Example Thanks log for Trizek (WMF).

I think we would only want to do that if the Thanks number was more than zero. Our designer is concerned that the Thanks log is likely too confusing for new editors though. Which I think might be true, but also not too many new users actually get Thanked, and maybe once a newcomer is Thanked then learning about the Thanks log is OK?

Ok, thanks again for all the previous feedback and guidance, we are definitely continuing to use that feedback to decide how to iterate on designs and scope our work! And here's a shameless plug to watch the Growth Updates page if you are interested in Growth team work. I post updates most weeks on what the Growth team is working on. Thanks!

Hello @John Broughton, @Pigsonthewing, @Mike Peel, @WereSpielChequers @Sdkb @Barkeep49 @TheDJ @NGC 54

Thank you all for previously providing feedback on the Growth team's Positive Reinforcement plans!

Impact module comparison

If you are interested, you can test the first part of Positive Reinforcement: the new impact module. You can opt in and view the new impact module by adding &new-impact=1 to your homepage URL.

Example for English Wikipedia:

https://en.wikipedia.org/w/index.php?title=Special:Homepage&new-impact=1

We are releasing this change to Growth pilot wikis (Arabic, Czech, Bengali, and Spanish) and we will complete a month long A/B test comparing the old impact module to the new impact module. If results look promising, we will release to more wikis once we've also addressed any issues or pain points surfaced by our pilot wikis and anyone who chimes in here.

If you have a chance to test, let me know what you think. Do you think newcomers will benefit from the new layout? Do you notice any bugs or aspects of the feature that are confusing? Thanks!

@KStoller-WMF: It looks good to me, but it is annoying that some lines span on 2 rows (at least in Romanian). P.S. The new version says that I have made only 1,000 edits, but the old (current) version says the truth (36,754 edits).

Thanks for testing!

Yes I see what you mean about some of the translated text spanning 2 rows for certain browser window widths. It's tricky to find a perfect solution there due to translations varying in length and since the module adjusts depending on the window size.

Thanks for noticing the 1,000 edit issue, hopefully we can find a reasonable solution there. T324286

I like the idea, but, as always, things could be improved:

• On the first page, the "1 of 2" would be better placed to the right of page title
• Showing "1 of 7,216" (or "1 of [an even larger number]) seems unduly daunting. Why not just "1 of 50" or "1 of 100"?
• When the first article is shown, the right arrow works (goes to the second), but the left arrow doesn't. Can you hide the left arrow?
• Showing the email address at the top of the page is, at least for more experienced edits, weird. Communicating by email is strongly, strongly discouraged, because it isn't consistent with the "everything should happen on-wiki" philosophy of wiki transparency. It's also unnecessary, unless I'm missing something major here.
• I'd prefer that clicking on an article open that article in a new window or tab, at least if not in mobile mode. That way, when I'm finished editing, I can close the window or tab, and return to where I was.
• It would be nice to be able to sort by the number of visits in the past 60 days, and/ or the date when the article was marked as needing copyediting. Or, perhaps better, the software should use a combination of the two of them to prioritize the articles that are initially present. (More visits is better; an older flag is worse.)
• I'm not convinced that copy edits "help articles be more professional and trustworthy". I'd replace that with "make articles more readable, and - if done after checking with sources - more accurate."
• The information "i" at the top right has two subheadings, one of which is "Filter by topic" But what is being done is NOT filtering. The heading should be "Select by topic". Filtering **reduces** the number of items shown. What happens when the user selects an additional topic is that the number of items shown (listed) **increases**.

Thanks for the feedback, @John Broughton!

It looks like this feedback is mainly about the Suggested edits feed, not the "Your impact" module, but this is all very valuable feedback that I'll review with our designer. We hope to have time to continue to iterate and improve on existing features in the coming year, so I appreciate the thorough breakdown of improvement ideas!

Showing the email address at the top of the page is, at least for more experienced edits, weird.

This is prominent to encourage newcomers to add an email address (as a way to improve new editor retention). However, I can see how it might seem odd to continue to display that so prominently once an email address has been added.

When the first article is shown, the right arrow works (goes to the second), but the left arrow doesn't. Can you hide the left arrow?

I've noticed this too. The left arrow is lighter to show that it's "greyed out" and inactive, but I don't think the contrast difference between those two arrows is enough to make that obvious. I'll chat with our designer about this and the other ideas you've mentioned. Thanks again!

Yeah the upper right panel is clearly not working right. For instance I have definitely had a longer streak than the 6 days it says. That said I think the layout is strong. I look forward to seeing what data is gained from the pilot wikis but I am hopeful.

Thanks for taking a look, @Barkeep49! We've definitely made some compromises in this initial version to ensure the data can be computed efficiently. Each section is only evaluating the last 1,000 edits: which works great for newcomers, but is problematic for established editors.

I'll work with the team to see if we can fix that Total edit count T324286.

And I think we'll need to add in some clarification to the "Longest streak" that for editors with over 1,000 edits, that the Longest streak will only reflect the last 1,000 edits. We could increase that number, but the larger the number the more likely we will run into performance issues.

Thanks again for taking a look and for the quick feedback! I'll be sure to follow up in this thread again once we have some data from Growth pilot wikis.

Is it only looking at mainspace edits to calculate those numbers?

@Barkeep49 Correct, we are limiting the metrics to what can be derived from the user's last 1,000 mainspace edits.

Overall, the module looks good!

Thanks everyone who provided feedback on the new Impact module, we made some changes to the new Impact module edit count based on your feedback. We don't have experiment results yet, so the new Impact module is still only released as an A/B test on Growth pilot wikis (Arabic, Bengali, Czech, and Spanish), but hopefully I'll have more information to share soon.

Post-edit message T322386

Newcomer notification T322435

In the meantime, the Growth team is shifting focus to the next part of the Positive Reinforcement project: the Leveling up features. These features are meant to help engage new editors and encourage them to try new types of newcomer-friendly suggested edits. Like all major Growth features, we will conduct A/B tests to carefully monitor the impact of this work.

After our first round of community feedback several months ago, we shifted our approach and decided to descope goal-based incentives, including the daily goal idea. Our focus is now more on encouraging users to try more difficult task-types as they are successful with easier tasks.

We also plan to test two types of notifications to see if they help reengage new editors and improve newcomer retention. If you want more details you can read more about current Leveling up designs and plans here, or you can view all of the associated Leveling up Phabricator tasks here.

@Pigsonthewing @NGC 54, @Barkeep49, @TheDJ - Previously you all mentioned concerns around some of the goal or badge-based incentives we were considering. You can see that we have cut that out of the Leveling up project. Are you more comfortable with this plan now? Do you have any further feedback?

Do you think post-edit messages and notifications will encourage newcomers?

If the experiment shows that these features are helping with newcomer retention, then we'll plan to make these features community configurable via the Growth configuration, so communities can control if the features are enabled and how they work. Do you have any thoughts on what you would want to configure?

Can you think of anyone else who might want to offer feedback on this project?

(Pinging @BusterD, @Sdkb, and @Clovermoss since we've been discussing new editor challenges recently).

This does address my previous concerns and I'm pretty excited to see what the A/B test results will be. My expectation will be modest but meaningful improvements (versus the larger improvements goal/badge might have achieved at a higher moderation cost)

As we plan this positive reinforcement project, we are interested in community thoughts on these questions (and anything else!)

• What has worked well on your wiki for motivating newcomers?
• What ideas do you think we should consider?
• What could go wrong and what should we try to avoid?

I'm glad to see work in this area—I think that all three ideas are likely to prove helpful. A few initial thoughts:

First, when developing ways to praise newcomers, I'd prefer to see approaches that are integrated into existing systems rather than introducing new ones. We already have the thanks button for small improvements and (at least at en-WP) Barnstars for more major things. Any new system on top of that would seem to me to be redundant, and the community might struggle to figure out how to fit it into our culture. Instead, I'd prefer to see functionality that perhaps flags contributions that might be worthy of thanks and makes it more convenient to hand it out. One piece of low-hanging fruit is phab:T51087, which would make it easier to keep track of who you've thanked as a mentor or what you've been thanked for as a mentee.

Second, one thing I've noticed newcomers are often very eager for is awards to put on their userpage. People just inherently like designing their profile. I'm not sure if that correlates with also wanting to edit articles or if many users just make their page and then quit, but it's something to be aware of. Things like the userbox I introduced here for completing the tutorial have gotten fairly wide usage.

Third, I've mentioned this before, but I definitely encourage improvements to the impact module to be done in a way that scales as editors become more experienced. Wikipedia has struggled for a long time with directing editor energy to the articles that are most popular with readers (as opposed to most reflective of editor biases) and valuing quality content contributions over high-volume gnoming. A good way to showcase impact that incorporated both pageviews and edit type could go a long way toward addressing those major issues. Plus, as newcomers grow into more experienced editors, it'll be nicer for them if the modules they started with continue producing meaningful data.

Thanks, @Sdkb, for taking the time to respond, and sorry for the delayed response!

I agree that we already have some existing systems that we should utilize when praising newcomers.  In the designs I’ve recently added to the Positive Reinforcement page, you can see we are currently considering ways to integrate Thanks and Wikilove into the newcomer homepage. I’ve added those images in the time since you made your comment.  What do you think?

I'd prefer to see functionality that perhaps flags contributions that might be worthy of thanks and makes it more convenient to hand it out.

We are still brainstorming ways to surface praise-worthy edits to mentors or other contributors, but we are thinking along this same line. How do you think we should flag contributions that might be worthy of thanks?

Thanks for bringing up phab:T51087, I can see how that would make the  [[Special:Log/thanks]] more useful.  For instance, in the design in this mockup, maybe that would allow newcomers to click on “View all thanks (4)” and then see exactly which edits they got thanked for.  Is this along the lines of what you were thinking?

Second, one thing I've noticed newcomers are often very eager for is awards to put on their userpage

This is a great point. Current designs include newcomer awards only being visible on the newcomer homepage, but awards and recognition are likely more effective if they can be shared or seen by others. We’ll keep this in mind as we start to refine the “Leveling up”  idea.  What do you think of an idea where newcomers could “port” awards over to their user pages?  Or do you think that should happen automatically?

I definitely encourage improvements to the impact module to be done in a way that scales as editors become more experienced.

I agree that ideally the impact module can be built in a way that is scalable and useful as editors become more experienced, but currently our team is most focused on new editors – the parts of their journey that happen over their first week or two on the wiki.  At what point in the lifecycle do you think a newcomer might want to start seeing a more advanced version of the impact module?

Thanks again for all of the valuable feedback! Please let us know if you have any further feedback now that more designs and details have been added to the Positive Reinforcements page.

Broadly, the positive reinforcement seems like it's on the right track.

For surfacing contributions that might be worthy of thanks, that could be tricky. If I were to search for candidate edits, I'd look for those from a newcomer that have a high ORES score and have been seen by an experienced editor who chose not to revert them.

For T51087, yes, that's generally what I'm envisioning, although I think it's important editors be able to navigate to a diff if they tap on an edit they were thanked for, as many editors will have made multiple edits to a page and it may not be clear which they were thanked for.

On porting, generally the lower friction something is, the better adoption it'll have, particularly among newcomers. So ideally, all someone would have to do is add a template to their userpage that'd then display their awards.

Re advanced version, that'd come into play as soon as the beginner version stops presenting useful data. My understanding is that currently it only checks your past 10 edits to look for popular pages, so as soon as someone passes that count, its usefulness starts diminishing.

I'm not sure any of the projects I'm familiar with (at varying levels) have figured out how to motivate editors. I know I was a person in earlier stages saying that we don't want new users to get "stuck" in certain tasks. The keyword there is stuck. If an editor only wants to, say, add links to articles and that's what makes them happy and they want to do that over some long period of time, that's great. It's just that, as you're finding here, that doesn't necessarily help with longterm engagement.

In terms of your ideas I think Impact has the most potential, especially because good impact tools could also help with editor retention. Obviously editor retention isn't the goal of this group, but pieces that could work those dual purposes are positive to me.

The leveling up piece is one that has caused problems on English Wikipedia as a motivator. It's certainly effective, but also produces quite a bit of effort that ends up requiring time and attention from other editors, and often the community at large, to deal with. So to the extent that you are going to do that, I would encourage it to be more like English Wikipedia's "Service awards" https://en.wikipedia.org/wiki/Wikipedia:Service_awards which have a time element involved so there's less of a motivation to reach a certain level all on its own.

I also think the personalized praise option has a lot of positive potential and like Sdkb's thoughts about barnstars, userboxes, and the phab feature request.

Thanks @Barkeep49 for taking the time to offer your feedback!

Yes, we hope the leveling up idea keeps users from getting “stuck” on one task but rather starts to expose newcomers to the vast range of possible tasks available to contributors.

I’m glad to hear you think the Impact improvements seem promising! We hope this project has a positive impact on new editor retention.  Retention is a key metric we will be tracking closely as we work through this project.

Could you tell me any more about leveling up causing problems on English Wikipedia? Was there a specific effort or tool that you’re thinking of?  Do you mean it required more time and attention from other editors due to bad edits?

I would encourage it to be more like English Wikipedia's "Service awards" https://en.wikipedia.org/wiki/Wikipedia:Service_awards which have a time element involved so there's less of a motivation to reach a certain level all on its own.

Including a time element isn’t currently part of the leveling up design, but might be something to consider.  

I’m glad to hear you think the personalized praise idea has potential. It seems like the main challenge we will face is in surfacing potentially “praise-worthy” edits to users who would be willing to offer praise or guidance. How do you think we could do that?

Enwiki's experience has been that if you make things into a game people will play it like a game. So the goal becomes the task and in accumulating volume to get that "reward" rather than maintaining quality standards. We have seen this in any number of patrol areas, including with the use of Huggle and more recently in the somewhat rocky rollout of RedWarn (now called Ultraviolet).

I agree with barkeep that goal based awards are easy, but can be problematic. We are here to make the encyclopedia, not for ppl to score as many points as possible in "add short descriptions" games. This seems very difficult to grasp for newcomers. That it is about collectively achieving something for the world. I do think that time based service awards like barkeep was saying are better at that. And note the verbiage: "service", you are doing something that is not selfish. Also "x thousand people have benefitted from your changes" might work, or things like "you helped 10 fellow editors". The encyclopedia and community building aspects need emphasis, not "x edits".

Thanks @TheDJ and @Barkeep49, you bring up good points. We definitely want to be careful about how we incentivize edits and tasks to make sure we find the right balance; we want to better onboard newcomers but ensure it's done in a way that is engaging and feels attainable to a wider audience.

I like the idea of being more service-oriented in the language we use and the awards we consider. I think it's also important to recognize that that sense of service and giving back to the community evolves over time and new editors might initially be more extrinsically motivated. Research indicates motivations for Wikipedia editors are multifaceted, and shift over time and experience. Newcomers are often driven more by curiosity and social connection, but certainly these motivations evolve if they stay engaged and continue to contribute.

Does this "New editor learning path" design seem like the right balance to you? If not, what would you change?

I don't think what you're suggesting plays to either curiosity (the way suggested edits do) or social connection (the way mentors do). It does, instead, play to extrinsic motivations of badges/rewards/status which is a real motivation that does work. The problem remains that if you're encouraging 60 easy suggested edits, the caution that an editor might feel on suggested edit 1 is different than they'll feel on 2 and different than 30 and 60. If they are doing them all right, that's great. If they make a mistake but have been motivated to get the badge you now have 60 edits that need cleaning up. As an example of the way extrinsic motivation can cause real problems for the English Wikipedia community, I'll note the community's ongoing struggles with #WPWP contests https://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Archive336##WPWP_is_back

Got it — that makes sense.  Okay, we’re going to keep this in mind as we take another look at the designs.

The Growth features have a "quality gate" that we allow communities to customize. Basically communities are able to set a maximum number of structured tasks that a newcomer can do in a day. This somewhat addresses the concern that we are setting goals that could end in a lot of edits that need cleaning up. But I realize it's not a perfect answer to that issue, and we definitely need to be careful about setting intentions that could lead to messy edits. Thanks again for the feedback!

@Barkeep49 @TheDJ Based on your feedback, we added a few more design ideas to the "Leveling Up" idea.

The first mockup is around setting some quality "gates" around the newcomer challenge. The first part of the quality gate idea to put a pause to limit further damage. This is similar to the existing limits we have in place for newcomer structured tasks.

The second mockup is to have a 'verification' timer so that even if a newcomer completes edits quickly, they will still need to wait to ensure edits aren't reverted before receiving the award.

We are still brainstorming what sharing an award would look like, but I certainly think @Sdkb is correct in the thinking that many newcomers are eager for awards or some positive recognition on their userpage.

Are we moving in the right direction with these new designs? Do you have any additional concerns or feedback?

These are definitely a step in the right direction. I find it interesting that in the new movement strategy forum (courtesy ping to @Xeno (WMF)) that they explicitly have gates before an editor can do certain things in the community https://blog.discourse.org/2018/06/understanding-discourse-trust-levels/

I think that setting daily goals could led the newcomers to make bad edits only because they want to achieve the goal. Maybe they could even start to edit mechanically, editing only with suggested edits, without getting to create articles, for example.

Also, there have to be a way to eventually encourage newcomers to create articles and edit without suggested tasks.

Thanks for the feedback, @NGC 54!

The Growth features include "Create a new article" as a "Hard" task that we initially try to steer newcomers away from, although we also provide some guidance about the task: "To successfully create a new article, you'll need to use many of the skills you can learn through completing some easier tasks. To learn more about how to create a new article, click here."

But I agree, eventually we want newcomers to learn to edit and create articles without needing suggested tasks. We hope to address this with some of the "leveling up" ideas we will focus on with the Positive Reinforcement project. Obviously we don't prevent newcomers from creating an article, but we have the option to "unlock" this task after a newcomer reaches a certain level of expertise. Do you have any thoughts on how many edits a newcomer should have before we "unlock" that "Create a new article" task? Do you think communities would want to customize that number to fit their wiki?

@KStoller-WMF: In order to unlock, maybe a few easy tasks, just a little medium tasks and at least one "Expand Article" task. But the unlocking method could mislead the newcomer to think that the article creation is possible only after completing the requested tasks. There aren't any other better solutions?

For multilingual newcomers, I think that Section Translation should work...

The most annoying thing when mentoring are the newcomers that come at Wikipedia to write about themselves or non-notable people (so there are issues with notability, COI, NPOV) or their company/business (so there are again issues with notability, COI, NPOV). They create articles and/or user pages about the topic. There have to be a way to tell the newcomers about what does a user page is and isn't (maybe some text in the help panel?). Also, there have to be some short guidelines about creating articles. These guidelines should discourage the users to write about themselves, and so on, and the new users should be aware at least of notability before creating an article, not after. But in the same time, the guidelines should not discourage the newcomers to create useful articles.

Currently, there is only an internal link to an article wizard. I do not think that too many newcomers read long pages.

Some examples: 1, 2, 3, 4. Examples can be found on the user talk pages of other mentors from ro.wikipedia.org, too.

For multilingual newcomers, I think that Section Translation should work...

Hello @NGC 54 - I just added this task and thought of you:

T321529 Design: adding a Translation task to the Newcomer Homepage

Feel free to add any feedback here or in that task if you have any. We are still in the early phases, but I'm hoping our designer can focus on that task soon.

And thanks again for all of the feedback! We plan to make some minor "Create a new article" UI improvements in the near term, and longer term hopefully that's something our team can focus on.

I don't know if this is the right place to post this, but I am rather concerned that the mentors proposed on the homepage are themselves relatively new users.

Hello Kudpung

It is up to communities to decide who can or can't be a mentor. Most wikis are on a "please volunteer" mood, as questions asked by newcomers are pretty basic.

Most mentor lists are Autoconfirmed protected. Only users with more than 500 edits and 90 days of presence are allowed to edit the list to add their name. This limit has been chosen because it is the highest available by default on wikis (except admin-protected). Any community is free to define conditions to become a mentor. Maybe it is a discussion to have with English Wikipedia mentors?

We are currently working on a new way to manage mentors lists. We are considering to let communities define by themselves the number of edits and the length of involvement to be a mentor.

Thanks for your reply Trizek. I didn't really want to interfere, but I was a bit surprised. I've actually not seen much mention of the Positive reinforcement project on the English Wikipedia. I used to be an admin for many years, but I've been in semi-retirement now for a couple of years. I think your project has potential. Keep up the good work 🙂

Hi @Kudpung, thanks for checking out the Growth features! And thank you @Trizek (WMF) for responding!

Although we've been researching and hoping to start on the Positive Reinforcement project for some time, it's just now starting to enter the development phase. We've been gathering feedback from our pilot wikis first, and we'll use this feedback to improve designs. We will then iterate on the features further once they are testable on pilot wikis. Once the features have been tested and improved on our pilot wikis, we'll start the process of rolling them out on more wikis and discuss the features in more detail on English Wikipedia.

I'm so glad to hear you think the project has potential! I really think the Positive Reinforcement project has the potential to help improve newcomer retention, but we'll be sure to have a data scientist analyze retention and other metrics to ensure our work is moving things in the right direction before we release it to more wikis.

If you want to stay in the loop on this project and other Growth projects, I post weekly Growth team updates and we also have a Growth Newsletter.

(I'll follow up more on the Mentor-side of your question in the other Mentor thread you started).