As I mentioned in the growth session at Wikimania, the Cantonese Wikipedia has independently come up with a plan for a bot which would embody a similar idea as the Newbie Dashboard, which would provide statistical feedback to encourage editors to return.
We would be interested in having the newbie dashboard (and the growth experiments in general, if they come as a package) on the Cantonese Wikipedia.
@Deryck Chan -- it was great to meet you at Wikimania. I hope your travels home went well. I'm glad you're interested in Growth features for your wiki. When @Trizek (WMF) is back at work next week, he can get in touch about how to proceed.
Cool, thanks Marshall. I'll be at another conference in Iceland next week so I don't think I'll be very active on-wiki then. yue:User:恐狼博士 (Dr. Direwolf) may be able to give you a faster response in the meantime. He's also an admin on the Cantonese Wikipedia, one of the original proponents of the bot, and a near-native English speaker.
Hello @Deryck Chan and @恐狼博士
I'm back from vacation and I'm really happy to resume work reading these good news about Cantonese Wikipedia. :)
All the instructions about getting the Growth team features are listed on Growth/Communities/How to get the Growth experiments on your wiki. You can start the process anytime after a community discussion that will have led to the decision to get the features.
Deryck, you've attended the Growth presentation so you know almost everything about the prototypes. But if anyone on Cantonese Wikipedia has questions, I'll be very happy to provide more information. Please ping me there.
One of our future steps is to provide task recommandations. So I would be curious to know more about Dr. Direwolf's bot! Can we have a longer description about its process? (Google translate is not always accurate for Cantonese, I'm afraid.)
Thank you again for your interest!
I'm slightly confused by the process instructions. What goes first - phab task or translations? And where does one go to localize e.g. Growth/Personalized first day/Newcomer homepage - I presume it's the interface, not the MediaWikiWiki documentation, that needs to be translated?
The Phab task is just a way to follow up on the entire process. When you create it, it gives you all the steps:
The link to interface translations covers all translations for Growth experiments, whatever the prototype.
We are currently planning a research study to implement the bot and investigate its effectiveness. Our plan is to recruit a group of participants from Hong Kong, give them basic instructions on writing Wikipedia articles, and then let them write. Some of them will be supported by the bot (experimental group), while some other won't (control group). We will then compare the two groups on multiple metrics. We are still doing ethics application, and I am not sure whether I am at liberty to share details about the bot with someone outside the research group.
@Deryck Chan What do you think?
@Trizek (WMF) Please don't get me wrong. I will be happy to work with you. But because I am not only person involved in the research project, I am worried that me sharing details about the bot without my co-workers' approval might displease them.
Aren't most of the details of our proposed bot already public? I don't think there's anybody on the team to displease if we published the details on-wiki properly (at meta:Research:Projects), and indeed I think we have a duty to update keep publicly readable documentation of our research as we make progress. Further, documenting our research on-wiki also prevents others from stealing our idea :)
Of course, we won't share anything that isn't appropriate for public sharing - e.g. any data about participants except their publicly logged edits and actions on-wiki, and anything that might sabotage the experiment if shared.
I intend to start a separate discussion about the "newbie statistics bot" later - I know User:Samat and the Hungarian Wikipedia community is also interested in making something similar. But let's cross that bridge when we come to it. @恐狼博士: Let's focus on our Cantonese Wikipedia bot as you have described in your university's research proposal, and worry about sharing the bot design after we start using it.
@Trizek (WMF) So here is the specification of the bot: The core idea of the bot is to gamify the Wikipedia-writing experience for contributors. Gamification is a commonly used concepts in fields such as pedagogy and management. It refers to using the principles of game design into non-game context. E.g., in game design, one of the principles is to make sure players have immediate feedback about their progress. Thus, a common practice in pedagogy gamification is to have students do online exercises accompanied by computer programs created to give immediate feedback.
The functions of our bots are as follows:
1. The bot identifies the users to be tracked. It will use an array to store the names of users it is tracking.
2. For each user, the bot proposes articles under 500 words for them to write. Group 1 participants will be recommended articles that are under the topics they list as their interests, while group 2 participants will be recommended articles that are not under their topics of interest. According to the flow theory, activities that are perceived to be relevant to oneself are more likely to induce flow (Shernoff et al., 2003).
3. Each user will earn points based on the quality of their contributions. At the end of each week, the points will be displayed to them on their user talk page, alongside tips and hints for improving their scores (flow theory, principle of feedback).
4. After the user has added a set milestone of word count, the bot will leave comments at the user’s talk page or the talk page of the article they are editing, to give comments. The comments will be randomly taken from a set of prepared comments, which are to be written by the researchers. The comments will be individualized to a degree, e.g., a comment that suggests an user to utilize more references will be more likely to appear if the article(s) written by that user has lower-than-average citations per unit word counts (flow theory, principle of feedback).
5. If an admin or editor edits an article by the focal user, the bot will prompt the admin or editor to give comments to the user (flow theory, principle of feedback).
6. If the user’s score has exceeded a certain threshold, the bot will prompt admins and editor to nominate him/her to become new admin and editor, so that he/she will be able to access new functionalities (unlockable contents are commonly used in game design).
7. The bot will keep doing 1-5, until a milestone (total word count of 30,000, according to the current plan) has been reached. After that, the bot will leave a final comment on the user’s talk page.
8. The bot will create a leaderboard, listing the names of the top 20 users who have the highest scores in that month, and send messages to the focal users informing them of their positions on leaderboard (principle of competition). Leaderboard is a common feature in gamification to provide opportunities for competition (e.g., Deterding, 2013; Dicheva et al., 2015; Zichermann & Cunningham, 2011).
9. In the context of Wikipedia, researchers cannot prohibit certain users from using the features of customizing one’s user page. Hence, the manipulation cannot be ‘whether the user can customize’. The bot can, however, instruct and prompt the users it keeps track of to customize their own user pages, thus creating more opportunities for these users to use customization features. For each user, the bot will record the number of edits and word counts he/she made to his/her user page (customization is commonly used in game design).
10. For each user, the bot will give him/her the names of under-bot users who edit similar article types as him/her and prompt them to contact each other to discuss potential opportunities for cooperation (principle of cooperation).
We are planning to do a field experiment with this bot and publish papers from it.
Thank you for the details! This approach goes deeper than the task recommendation project we have started to work on.
Concerning the step 2, how do you identify the topics of interest? We could help, through the Welcome survey.
@Trizek (WMF) Our current plan is to host a workshop for participants (as mentioned, we will give them basic instructions on writing Wikipedia first). We plan to give them some questionnaires to fill in at the workshop.
@Trizek (WMF) Do you think this idea is viable? If it is, maybe we could try it in other versions of the Wikipedia as well.
@Trizek (WMF) Speaking of that, maybe we could instead conduct a larger scale study using English Wikipedia. It isn't absolutely necessary for us to run the study in the Cantonese Wikipedia.
I think it is wise to try on a small wiki before running it on a larger one. This is how the Growth team started working on its experiments. :)
@Trizek (WMF) That is understandable. However, due to what have recently been occurring in Hong Kong, some of my coworkers have informed me that recruiting participants from there might be difficult until end of the year. Therefore, we are planning to recruit participants from other regions, so that in the worst case scenario, we will still get to conduct the study, and in the best case scenario, we will have data from multiple cultures and thus enable cross-cultural analysis.
@Trizek (WMF) Do you think there might be other small wikis who are interested?
If you get the Welcome survey deployed on your wiki, you would be able to get usernames with topics of interest, from people who create an account on your wiki. It would solve the face-to-face issue.
Concerning other wikis, I don't know. At the moment, the wikis I'm in touch with are interested by getting the Growth features or currently have them as an experiments with a A/B test. I think the best way for you is to run your experiment on your wiki.
