성장/개인화된 첫 날/구조화된 작업
|
Newcomer tasks
 Recommend tasks to newcomers that help them start editing
|
이 문서는 성장 팀의 "구조화된 작업"에 대한 설명입니다. 이 프로젝트는 새 사용자 작업과 새 사용자 홈페이지 프로젝트의 연관 프로젝트입니다. 이 문서는 프로젝트의 주요 요소, 디자인, 결정 사항 등을 나열하고 있습니다. 대부분의 순차적인 업데이트는 성장 팀 업데이트에 게시되며, 대규모 업데이트가 이곳에 기록됩니다.
This page is about the general "structured task" concept, with some discussion of specific task types that we could build. Following this general discussion, the team began designing and engineering on those specific task types. Those task types have their own project pages, where most new information is being posted:
현재 상태
- 2020-05-01: 초기 노트 계획과 기록
- 2020-05-17: begin community discussion
- 2020-05-29: initial wireframes
- 2020-08-24: week of planning meetings
- 2020-09-08: call for community discussion on latest designs
- 2020-10-21: user testing of desktop designs
- 2021-05-27: We deployed the "add a link" feature to our four pilot Wikipedias (Arabic, Czech, Vietnamese, Bengali).
- 2021-07-21: continue to scale "add a link" to more Wikipedias (T304110)
- 2022-08-20: start work to address patroller feedback about structured tasks (T315732)
- 2022-09-02: publish Newcomer task edit type analysis
요약
성장 팀은 2019년 11월에 새 사용자들이 홈페이지에서 편집 추천을 받을 수 있는 새 사용자 작업 프로젝트를 배포하였습니다. 2020년 4월 기준으로, 추천 편집 문서는 이미 숙련된 사용자가 유지보수 틀을 추가한 문서를 통해서만 제공되므로, 문서의 어떤 부분이 특별히 주목해야 하는지 알려주지 않습니다. 문서의 어느 부분을 편집해야 할 지 알려주지 않아도 많은 새 사용자들이 많은 생산적인 편집을 만들고 있다는 것에 기쁩니다. As of April 2020, the suggested articles are sourced only from articles that have maintenance templates applied by experienced editors, which do not give newcomers particular direction on which sentences, words, or sections specifically need attention. Despite this lack of direction, we are happy to see that newcomers have been making productive suggested edits.
유지보수 틀이 새 사용자가 편집에 적응하기 위한 다양한 종류의 편집을 공급해 주기는 하지만, 이러한 문서들은 새 사용자가 편집하기엔 너무 광범위하거나 모호할 수 있습니다. 특히 모바일 기기의 작은 화면이 시각편집기나 원본 편집기가 새 사용자를 혼란스럽게 할 수 있습니다.
따라서 저희는 "구조화된 작업"이라는 작업을 실험해 보고자 합니다. 이러한 작업은 편집 작업을 새 사용자가 쉽게 완수할 수 있는 여러 순서의 작업으로 나누는 것을 목표로 합니다. 안드로이드 앱 팀과 언어 팀의 예시를 기반으로, 이러한 편집을 쉽게 하면 더 많은 편집을 할 수 있을 것으로 생각됩니다. 이러한 구조화된 작업은 새 사용자 작업 프로젝트의 일부로써 새 사용자에게 제공됩니다. This is about breaking down editing workflows into a series of steps that newcomers can accomplish easily. Following the successful examples from Android and Language team work, we think these types of edits will be easier for newcomers to do and easier to do on mobile, helping more newcomers do more edits. These structured tasks would be accessible to newcomers as part of the newcomer tasks project.
배경
편집은 복잡합니다
성장 팀의 경험을 통해, 새 사용자는 최초 편집 순간에 편집을 계속할 지, 편집을 중단할 지 결정하는 것으로 추측하고 있습니다. 새 사용자가 빠르게 편집을 저장하고 긍정적인 경험을 할 수 있다면 남아 있을 확률이 높죠. 하지만 위키백과 기여는 (거의 모두 다) 복잡하고, 이러한 요소가 빠른 성공을 방해합니다. 예를 들어, 문장 하나를 추가하려면 거의 12개에 가까운 작업을 해야 합니다. We believe that newcomers want to stay when they can quickly make an edit and have a positive experience. But contributing to Wikipedia -- almost any type of contribution -- is complicated, and this makes it hard for them to succeed quickly. For instance, there are about a dozen steps required in order to do something as simple as adding a single sentence to an article:
- 올바른 문서를 검색합니다.
- 추가하려는 내용이 문서에 이미 존재하는지 알아냅니다.
- 문장을 추가할 문단을 찾습니다.
- 문서를 편집하기 시작합니다.
- 올바른 위치에 내용을 집어넣습니다.
- 인용 버튼을 누릅니다.
- 출처로 돌아가 링크나 인용 정보를 얻습니다.
- 인용 폼을 채우고 저장합니다.
- 문서를 게시하기 위한 절차를 시작합니다.
- 편집 요약을 적습니다.
- 문서를 저장합니다.
시각편집기나 원본 편집기를 사용하는 새 사용자는 이러한 절차가 무엇인지 모르고, 어떤 순서대로 해야 하는지도 모르고, 어떤 버튼을 눌러야 하는지도 모릅니다. 즉, 이러한 절차는 "구조화"되지 않았다는 것입니다. 너무 복잡해 보여서 그냥 포기하고 다른 일을 하러 갈지도 모릅니다. 여러 실수를 하고, 다른 사람으로부터 부정적인 피드백을 받을지도 모르죠. 이게 이 프로젝트가 하려는 일입니다. "어떻게 하면 새 사용자가 올바른 순서를 따르도록 인도할 수 있을까?" In other words, their experience is not structured. They may just be overwhelmed and leave. Or they may use trial-and-error, make a mistake, and get negative feedback from experienced editors. That's what this project is about: how might we help newcomers step through these workflows in the right order?
Building on knowledge from other teams
Adding structure to editing workflows has been part of the Wikimedia projects for a long time. Some examples include:
- HotCat: lets users choose categories to add to articles with a few clicks, instead of manually editing the wikitext.
- Commons Upload Wizard: breaks the process of uploading media to Commons into a series of a simple steps.
- Citoid: available in the Visual Editor, this breaks down the process of adding a citation into steps that include algorithms to automatically produce the citation text and template.
Most recently, the idea of "structured tasks" has been working well on the Wikipedia Android app and in the Content Translation tool. We're inspired by their work.
With their "suggested edits" project, the Android team broke down the process of adding a title description to a Wikipedia article into one easy step of typing into a text box. They have since done the same with translating title descriptions across languages. In order to do the same tasks without a structured workflow, users would have to go to Wikidata and go through several steps to make those same edits. The team learned that this method works: many Android users make hundreds of these small contributions.
The Language team built the Content Translation tool, which does several things to structure the process of translating an article. It offers a side-by-side interface built for translations, it breaks the translation down into sections, and it automatically applies machine translation algorithms. Though Wikipedians could translate articles before the existence of the tool, the number of manual steps required made it very difficult. This tool is successful, with hundreds of thousands of translations completed. We learned that when translating an article is broken down into steps, with rote parts (e.g. running machine translation) taken care of automatically, more articles get translated.
The Growth team is thinking about applying these same principles to content edits in articles, like adding links, adding images, adding references, and adding sentences.
-
The Wikipedia Android app offers "suggested edits", such as translating title descriptions, which users can do in one easy step, instead of going through Wikidata. -
The Content Translation tool breaks down the process of translating an article into one section at a time, with automated assistance from machine translation algorithms.
A structured task sketch
The best way to explain how we're thinking about structured tasks may be through showing a quick sketch. The first structured task we've thought about is "add a (wiki)link". But the same ideas could apply to structured tasks for "add an image", "add a reference", or even "add a fact".
In the newcomer tasks feature, lots of newcomers complete "add a (wiki)link" tasks -- in which they add internal blue links in articles that don't have many. This seems like a simple editing task to get started. But we think that many newcomers may not understand how to go through the steps of adding a link and may not know which words to make into links. We're imagining a workflow that walks them through it, step-by-step, with the assistance of an algorithm that can guess which words or phrases might make the best links.
In the sketch below, the newcomer arrives on an article, and is given a suggestion of a word that might make a good (wiki)link. If they agree that it should be made a link, they are walked through the steps of making the link. This will hopefully teach them to add links on their own in the future -- and perhaps they'll enjoy continuing to receive these algorithmic link suggestions. Regarding the algorithm, the WMF Research team has done some preliminary work that makes us confident that such an algorithm is possible.
In thinking further about this, we sketched a second idea. Instead of being aimed toward teaching the newcomer to add links using the visual editor, this next workflow lets the user quickly confirm or reject recommendations from the algorithm, directly editing the article. While it does not teach them how to add links via the editor, it might help a newcomer edit at high volume, and might be a better fit for a user who is trying to be productive with simple tasks while they are on the go. Or perhaps might be a good fit for users who only are interested in very simple edits, similarly to how the Android app has many editors who only want to write title descriptions.
In thinking about structured tasks, it looks like this might be a big question: should workflows be more aimed toward teaching newcomers to use the traditional tools, or be more aimed toward newcomers being able to do easy edits at higher volume?
Why this idea is prioritized
We think that quickly making productive edits is what leads to newcomer success. Once they've done some edits, the rest of the wiki experience quickly becomes richer. Newcomers can then see their impact, get thanked, ask informed questions to their mentors, create their userpage, etc. Therefore, we want lots of newcomers to make their first edits as soon as possible. We have already seen from the newcomer tasks project that many newcomers are looking for easy tasks to do. But we also have observed these things:
- Only about 25% of the newcomers who click on a suggestion actually edit it.
- Only about 25% of those who do a suggested edit do another one.
- There are a handful of newcomers who really thrive on suggested edits, doing dozens of them every day. This shows the potential for newcomers to accomplish a lot of wiki work.
- In live user tests, when newcomers are told to copyedit an article or add links to an article, they frequently want to know exactly which sentence or words need their attention. In other words, attempting to edit the full article is too open-ended.
Taking these points along with the experiences described above of the Android and Content Translation teams, we think we could increase the number of newcomers editing and continuing to edit by structuring some of the content editing workflows in Wikipedia.
Opportunities with structured tasks
When we break down editing workflows into steps, we call them "structured tasks". Here are some of the possible benefits we think could come from structured tasks:
- Make it easy for newcomers to make meaningful contributions.
- Develop editing workflows that make sense for mobile. Mobile design principles tell us that users should see one step at a time, not a complicated workspace.
- Let newcomers increase their skills incrementally. They could take on successfully more challenging types of tasks.
- Let people find an editing experience that fits them. By giving newcomers a feed of structured tasks, they could find the type of tasks that they prefer.
- Perhaps similar workflows could be opened to experienced editors in the future.
Concerns and downsides to structured tasks
Whenever we add new ways for people to edit Wikipedia, there are many things that can go wrong:
- By making editing too quick and easy, we may attract vandals, or users who don't apply enough care when editing.
- Giving newcomers simple workflows may keep them from learning the traditional editing tools, which are essential for doing the most impactful wiki work.
- Structured tasks may not be good at accounting for differences across languages, idiosyncrasies with wikitext, and could cause other kinds of bugs.
- Algorithms that surface structured tasks may not be accurate enough, and falsely encourage newcomers to complete edits they shouldn't.
Community discussion
In May 2020, we conducted discussions with community members in six languages (English, French, Korean, Arabic, Vietnamese, Czech) about the above ideas for structured tasks. The English discussion mostly took place on the discussion page here, with other conversations on English Wikipedia, and local language conversations on the other five Wikipedias. We heard from 35 community members, and this section summarizes some of the most popular and interesting thoughts. These discussions heavily influenced our next set of designs.
- Community members were generally positive about the potential for structured tasks to help newcomers start editing. But it was also a widely expressed view that it's important for newcomers to be introduced to the conventional source and visual editors during the process. Community members want to make sure that newcomers are not siloed in a separate editing experience, and that they can find their way to more valuable edits.
- The Czech community talked about ideas for how the structured tasks can place inside the visual editor, so that newcomers can start getting used to being in the editor. Perhaps the editing tools that are not needed for the structured task can be grayed-out.
- Community members asked why we are choosing "add a link" as our first structured task, as opposed to higher-value types of edits. We talked about how this task is one of the easiest for us to build, which will help us prototype and learn from structured tasks sooner, and how it is a comparatively low-risk task, with fewer opportunities for newcomers to damage articles.
- Several communities mentioned that spelling corrections would be a particularly valuable task, and we talked about technical options for how to generate lists of potential spelling mistakes. See these notes for more details.
- We also talked about whether reverting vandalism is a good fit for newcomers. It doesn't seem like the answer is clear, and this will have to be discussed more in the future.
- An idea that was mentioned multiple times is how to "step newcomers up" to progressively more challenging tasks, perhaps while giving them rewards for successfully completing easier ones.
Types of tasks
There are many different editing workflows that have the potential to become structured. We began to list workflows when we first designed the newcomer tasks workflow here, and we have since narrowed down to a shorter list of task types that seem best suited to being structured. The table below contains that short list, ranked in a potential priority order.
| Potential priority | Task type | How it might work | Advantages | Concerns |
|---|---|---|---|---|
| 1 | Add a link | For articles without enough wikilinks, an algorithm (existing) suggests words or phrases that should become wikilinks, and the newcomer accepts or rejects the suggestions. | Linking is a quick and easy way to edit, and has low potential to damage articles. | Understanding when to add a link takes judgment, and we don't want articles to be overlinked. It is also not the most valuable type of edit. |
| 2 | Add an image | For articles without an illustration, an algorithm (potential) suggests an image from Commons. This might be a simple algorithm that just looks at what images are used on that article in other languages. The newcomer decides if the image belongs, and where in the article to add it. | Good images make a big difference in an article, and newcomers are interested in adding images. | Adding the wrong image to an article could damage the article in a very visible way. |
| 3 | Add a reference | Some sentences or paragraphs clearly need citations. An algorithm (in development) would point out which sentences likely need suggestions, and the newcomer would seek sources to add as citations in a step-by-step workflow. | References are of clear importance to the core of the encyclopedia. | This task may not be exciting to newcomers. They may also struggle to find and use sources without guidance. |
| 4 | Copyedit | Using open-source spellcheck dictionaries and code, or using Wiktionary, identify likely misspelled words, and point them out to newcomers, who can use the visual editor or wikitext editor to fix them one at a time. | Clearly valuable and needed in any wiki, satisfying to newcomers. Helps them start editing the main text of articles, as opposed to peripherals parts of the article. | Scaling to any language may be difficult, depending on the availability of good spellchecking algorithms. |
| 5 | Add a section | An algorithm detects when an article could use additional sections, based on the kinds of section headers that similar articles have (e.g. all biographies of scientists tend to have "Publications" sections). The newcomer is walked through producing a well-referenced paragraph. | Real content additions that could help close knowledge gaps. | A much more challenging task than the others, requiring many wiki skills to be used together. May produce low-quality content. |
Prioritizing "add a link"
The Growth team currently (May 2020) wants to prioritize the "add a link" workflow over the other ones listed in the table above. Although other workflows, such as "copyedit", seem to be more valuable, there are a set of reasons we would want to start first with "add a link":
- In the near term, the most important thing we would want to do first is to prove the concept that "structured tasks" can work. Therefore, we would want to build the simplest one, so that we can deploy to users and gain learnings, without having to invest too much in the first version. If the first version goes well, then we would have the confidence to invest in types of tasks that are more difficult to build.
- "Add a link" seems to be the simplest for us to build because there already exists an algorithm built by the WMF Research team that seems to do a good job of suggesting wikilinks (see the Algorithm section).
- Adding a wikilink doesn't usually require the newcomer to type anything of their own, which we think will make it particularly simple for us to design and build -- and for the newcomer to accomplish.
- Adding a wikilink seems to be a low-risk edit. In other words, the content of an article can't be as compromised through adding links incorrectly as it could through adding references or images incorrectly.
Notes on "copyedit"
In conversations with community members on this project's discussion page, many people brought up the question of how to make a structured task around copyediting. Correcting spelling, grammar, punctuation, and tone seemed to everyone to be a clearly useful task that should be prioritized. The Growth team initially shied away from this workflow because of scaling concerns: even if we were able to find or develop an algorithm that could reliably find copyedits in one language, would we be able to do that in dozens of other languages?
We began to learn more about this by talking with User:Beland, who developed the "moss" script for English Wikipedia's Typo Team. We wanted to understand how the process works, and what it might look like to do something similar in other languages. In short, it sounds like the most promising avenue is through existing open-source spellcheckers and dictionaries. Two examples are the aspell and hunspell libraries. Below are our notes from learning about "moss" with User:Beland.
- Prospects for doing something similar in other languages
- A process like this should theoretically work in other languages, given that other languages also have Wiktionaries and open-source spellcheckers.
- But it would not be possible to deploy in a new language without native speakers validating it. There would likely need to be customization for many languages.
- Likely more challenges for languages without word segmentation (e.g. Japanese).
- Likely more challenges for agglutinative languages.
- Different projects have differing manuals of style, which may cause issues.
- If an algorithm is performing poorly, it should always be possible to change its thresholds so that it identifies fewer potential errors, but with higher confidence.
- How does moss work?
- Download the dump files of all of English Wikipedia every two weeks.
- In order to cut down on false positives, remove templates and everything inside quotation marks, etc. Only want to work on the main text in the article: the things written “in Wikipedia’s voice”.
- Check that every word is in English Wiktionary.
- Uses Python NLTK (natural language toolkit) for word segmentation.
- Looks at edit distance to classify misspellings. e.g. “T1” is one edit distance (95% precision). Also classifies “TS” whitespace errors.
- Also includes an English open-source spellchecker to narrow the search space so that the algorithm can run faster.
- He has also started trying to add grammar rules (e.g. identifying passive voice), but that’s more experimental, and much more difficult than spelling.
- At the end of the process, it produces a list of articles and likely typos. The user opens the article and searches for the likely typo.
Many copyedit requests are also editors whose native language is not English, asking for English polishing. See WikiProject Guild of Copy Editors.