成长/聚焦服务台

成长

成长的功能

使用的工具

For communities

最新进展

增長团队

于Phabricator查看
任务T206711

维基媒体基金会的项目 Focus on help desk Make it easy for new editors to ask questions 群組： Growth 开始： 2018-10-01 成员： Stephane Bisson (software engineer), Benoît Evellin (community relations specialist), Elena Tonkovidova (QA engineer), Kosta Harlan (software engineer), Morten Warncke-Wang (data analyst), Rita Ho (senior user experience designer) 积压工作： #T206711 主导： Roan Kattouw 管理： Ryan Kaldari (engineering), Marshall Miller (product), Jazmin Tanner (program) 更新： Most updates posted here.

本页旨在描述增长团队在"聚焦服务台"项目上的工作，還包含了主要的资产、设计、和决定。

大多数的增粮式进展更新會发布在一般的成长团队最新進展页面上，而一些大型的或更详细的最新進展会发布在这里。

任何人都可以在测试维基百科中创建一个账户，在他们账户的偏好/编辑页面中打开帮助面板，然后在任何页面上点击"编辑"，试用這個真实的帮助面板，这是我们为支持这个项目而建立的功能，會在这个页面上详细的描述。

截至2019年9月，我们已经完成了对实验的分析，显示帮助面板对新手的激活或保持「没有」产生任何影响。 因此，我们将不再反覆改進该功能，而是专注于前景看好的新手主页和新手任务功能。 实验结果的细节请看这里。

法语维基百科帮助面板的截图。

帮助面板关闭时的屏幕截图。

向服务台提出问题的屏幕截图

当前状态

• 2019-09-30: 已完成了帮助面板的实验，并发表頭條新聞的结果。
• 2019-07-22: 在一次A/B测试中，帮助面板部署在阿拉伯语维基百科中，有一半的新来者没有收到它。
• 2019-03-26: 发表了最初的领先指标报告 。
• 2019-03-04: 帮助面板在帮助、帮助讨论、维基百科、维基百科讨论，用户和用户讨论命名空间中的阅读上下文中显示。
• 2019-02-28: 在一次A/B测试中，帮助面板部署在越南维基百科，其中一半的新用户没有收到它。
• 2019-02-14: 为治疗群组部署在捷克和韩国的维基百科中的搜索能力。
• 2019-01-11: 在一次A/B测试中，帮助面板部署捷克和韩国维基百科，其中一半新用户没有收到它。 要查看这些wiki中帮助面板的问题，请参阅他们的求助区：
  • 捷克语求助区
  • 韩语求助区
• 下一步：我们正在研究一个增粮式的功能，鼓励用户們提出问题，用以改善他们的搜索结果，和增加能讓他们去阅读帮助服务台的答复的可能性。

实验结果（2019年9月）

• 帮助面板于2019年1月首次部署到新手，截至2019年9月，我们现已完成数据分析以确定其影响。这只是一个简单的总结，更深入的分析即将推出。
• 总之，帮助面板「没有」显示出激活率（用户是否进行第一次编辑）或保留率（用户是否再次返回编辑）的增加。 虽然这个结果令人失望，但我们的团队还有其他几个有希望的功能，我们将继续进行试验。
• 帮助面板「确实」导致编辑量（所做的编辑数量）的轻微变化，但结果是矛盾的。它在新手的第一天就增加了编辑量，但在随后的两周内却减少了。
• 总的来说，我们目前不知道为什么帮助面板会导致这些结果，然後我们有很多理论可以讓我们继续思考。例如，在我们希望帮助面板通过提供他们如何做的信息可以鼓励新手进行第一次编辑的同時，也有可能因为它向新手展示了大量复杂的文档，这很可能是无聊的或令人害怕的，反倒是「阻止」了一些新手进行编辑。
• 虽然提高激活率和保持率是我们团队的最终目标，但我们也观察到大量的新手打开了帮助面板（20%），并在打开后与之互动（50%）。这告诉我们对帮助面板所提供的帮助是有需求的。
• 综合来看，这些结果令人失望--我们当然希望帮助面板对指标产生积极影响。 虽然我们对如何改进帮助面板有很多想法，但我们决定在可预见的未来不再反覆改進和开发帮助面板，而是将注意力放在新手主页和新手任务上。 这些项目有前景看好的早期效果，而且比帮助面板接触到的新手要多得多。 帮助面板也将作为新手任务项目的一部分，并将继续作为成长团队正在建设的新手体验的一部分。 我们将来可能会更深入地重新审视它。

摘要

新的编辑者在开始编辑时，普遍面临技术、概念和文化方面的挑战。 他们有具体的问题，并且可以从人类编辑者回答他们的问题中受益。 大多数的wiki都有一个服务台，新的编辑者可以在那里提问，这种场所已经被证明是有效的，特别是像英语维基百科中的茶館（见本文的影响分析）。 可是问题是，大多数新编辑者根本不知道或找不到服务台 -- 尽管他们经常在他们的谈话页上收到的"欢迎模板"，是带有链接的。

我们有两个假设：

• 新手很容易提出问題之时，則更有可能會寻求帮助。
• 新手知道去哪里寻求帮助，則更有可能會完成编辑。

因此，"聚焦服务台"项目包含两部分：

1. 邀请新的编辑者到服务台。对于如何邀请他们，我们有四个想法：在他们的谈话页上发布、使用横幅、通过电子邮件、或者在编辑环境中添加一个链接。 我们决定通过在他们的编辑经历中添加一个清晰明显的链接来邀请新的编辑者到帮助台，我们称之为"帮助面板"。这是有下列原因的：
   • 我们从新编者经验研究中了解到，能夠“在上下文中”获取帮助是很重要的。在用户需要时會提供一个链接以便获得幫助，并且是在目前没有明确方式可以获得帮助的地方获得帮助。
   • 我们已經公开了一些问题，這些問題都是围绕「在上下文中的问题或聊天」的想法，这将帮助我们了解到是否有这种功能的需求。
2. 一旦我们成功地推动新编者到服务台，就对服务台本身进行改进，使其成为更好的提问场所。 这可能包括在页面顶部列出新的问题、或是在新编者的问题已被回答之時通知他们。

增长团队在2018年第二季度（2018年10月-2018年12月）首先建立了第一部分（帮助面板），并于2019年1月11日在捷克和韩国的维基百科的一个受控的A/B测试中部署了它。 这项实验将需要到2019年7月才能完全完成。 随着结果的出现，我们可能会给该功能增加额外的能力、将它部署到更多的wiki中、或者决定开始第二部分（对服务台本身进行改善）。

對比評測

我们团队的设计师重新探討了其他平台（如SurveyMonkey、美国运通、Codecademy）向用户提供上下文幫助的方式（详情见工單T206713）。我们认为我们可以从其他软件中学习一些最佳做法，尤其是当我们在许多不同类型的软件中看到相同的模式。即使我们併入了其他软件的想法，我们仍然会确保维基百科独特的开放性、清晰性和透明度的价值得以保留。比较结果显示在这疊幻灯片，其主要的收获是：

上下文幫助的用户界面

• 上下文帮助通常以屏幕右下角的粘性按钮或标签的形式呈现
• 点击帮助通常会打开一个用户界面上方的粘性面板，提供一个或多个帮助选项
• 用户通常能够在上下文帮助面板中搜索和查看首要的帮助主题
• 上下文帮助也会将用户链接到专屬的/主要的帮助中心/论坛。
• 一般的/网站范围内的帮助也通常是一个主要导航选项，而且还可以访问用户界面内的聊天/反馈。
• 隐私和安全的指标也通常會显示在拓展的帮助用户界面中
• 移动（app/网頁）版本通常没有可在上下文中访问的帮助，而是单独打开、作为一个主要的导航的项目。

提供上下文中帮助的类型

• 链接到帮助中心/帮助台
• 针对该特定情况列出的最主要的常见问题
• 即时对话
• 视频/教程
• 導覽链接
• 以前帮助历史的查询
• 社区论坛--在一些地方使用，那里对响应时间和"服务"水平的即时性期望较低。
• 「提报隐错」--供用户发送关于意外行为/隐错的反馈，通常可以选择包括用户界面的截图。隐含的期望是用户应该"不要期望得到回应。

帮助中的聊天/问答的功能

• 当没有"即时"代理可用时，用户能够透過聊天面板电子邮件提问。
• 快速响应窗口，會提供反馈給用户，以及用户可以在哪里等待答复
• 聊天的声音和语气是親切和優雅的，有时在聊天中会有表情符号和gif圖的选项。
• 用户可能會期望有某个服务水平，那是因为聊天通常由付费客户的服务代表所提供。
• 會清楚显示出隐私的链接和信息，與如何管理聊天记录相關的
• 在特定的語境中的常见问题/主题通常在會在同一个用户界面中呈现，以作为低参与度高帮助度的用户的另一個选择。
• 有时会先提供自动答案/"答案机器人"，在用户們选择发布他们的问题之前，尝试先解决问题。

以下部分可能会在未来几周内发生重大变化，或者它们过于技术性或与项目理解不太相关。我们决定不翻译它们。

MediaWiki类似功能的回顾

在我们讨论帮助面板的想法时，几位維基媒體基金會的工作人员提醒我们，过去曾有过两个听起来很相似的功能：MoodBar和文章反馈。这些（虽然已不再使用）功能都給了用户一个提供信息的上下文工具，因此我们认为我们可以从那些功能裏吸取经验。这项工作的细节在工單T206714上，而主要的收获是。

Screenshot of Article Feedback Tool

文章反馈工具

• 账户创建和编辑激活的潛力 - 开始编辑的匿名用户在寻求帮助时可能会被提示创建一个账户，以便他们能够获得"帮助团队"的反馈。
• 在允许提交自由文本的意见和问题之前，提供更多的前期帮助选项。 - 根据文章反馈工具的研究结果，太容易提交非结构化和断章取义的文章评论会导致給版主帶來很大幅度的噪音。 虽然我们的假设是，正在编辑上下文當中并为特定任务寻求帮助的用户們應該不太會提交无用的问题，但预先提供更多自我导向的帮助替代工具可能可以进一步减少数量。
• 为评论提供上下文的机制 - 在AFT中的评论缺乏上下文是编辑们不满意的一个主要问题。对于上下文的帮助，如果提交问题或评论的用户能够指出他们在文章或用户界面的什么地方需要帮助，那将是最理想的--例如，能够与他们的问题一起提交一张截图，或者能够突出与他们的评论有关的部分。
• 将互动变成实际的编辑 - 这个工具为有经验的编辑者创造了一个单独的工作队列去處理，他们无法通过他们的观察清单或最近变更来管理，目前策劃通过现有的程序来删除破坏性内容。重要的是，我们创造的任何额外流量都可以通过现有的系统进行管理，如果与我们的功能之間的互动是以wiki页面上的貼文形式进行，这是最容易做到的。

Screenshot of MoodBar

MoodBar

• Capturing email for responses is an effective option – As shown in the MoodBar experiment, email notifications is an effective means of engaging new editors.
• Clear, prominent call to action for new editors to ask for feedback/help - per high level findings, the addition of a tooltip drawing attention to the MoodBar significantly increased its use.
• Tracking the perceived helpfulness of responses is important - It would be useful for us to track similar new user perceptions of feedback, since a poor experience from receiving unhelpful advice has been shown to negatively impact editing.

Design for help panel

Our evolving designs can always be found in this interactive prototype. These clickable mockups also contain other design ideas for future iterations of the help panel, although some elements in those mockups may be obsolete. To see the latest version of the help panel, edit a page in Beta or Test wiki after having turned the help panel on in your preferences.

The "Comparative review" and "Review of similar MediaWiki features" were critical in the design process because they helped us "dream big" to explore the space of what the help panel could eventually become. Our team mocked up and discussed many ideas that probably will not become part of the help panel unless we see that earlier, simpler versions are successful. First, we are going to work on an initial version of the help panel to be deployed in January 2019.

Initial version

In the initial version of the help panel, we want to answer these questions:

• Will newcomers click on a clear option to get help during the editing experience?
• Do newcomers seem more interested in reading content to answer their own questions, or in asking a question for others to answer?
• Will newcomers take action to write and post a question to the help desk?
• Will the presence of the help panel increase new editor retention?

Therefore, we have decided to include these elements in the initial version, detailed in T206717 and T209318:

• A call to action in the editing context.
• A panel that opens containing links to helpful existing pages.
• The ability to ask a question from that panel, and for that question to be automatically posted to the wiki's help desk.
• The ability to add an email address to their account if the user does not already have one, and to modify their email address if it is not yet confirmed (and then to receive a confirmation email upon submitting).
• The ability to turn off the help panel by clicking through to Preferences.

The mockups below show initial designs for that functionality as of November 2018 (to see the most up-to-date designs, use this interactive prototype). Right now, we will show this only to newcomers (meaning new account holders who are not auto-created). We also know that the wording in the feature is very important here, because it is critical that newcomers understand where and how information they write will be posted, and when and where they can except a reply. These mockups only contain some initial drafting of the wording, and we'll continue to refine it. We are hoping for strong ideas from community members, so please post any ideas on the talk page.

• 
  1) Mockup of initial call-to-action on desktop and mobile
• 
  2) Mockup of help pane initial state on desktop and mobile
• 
  3) Mockup of asking a question in help pane
• 
  4) Mockup of confirmation received after asking a question
• 
  5) Mockup of a question having been automatically posted to a help desk

Business rules for initial version

Below are the current rules we're using to determine who receives the help panel and how it works. These may change as our team and the community continues to discuss and learn from the feature.

• Users: the help panel will be a feature that can be turned on and off in user preferences. When it is deployed in January, it will be turned on to only newly created accounts (not auto-created) from the deployment date forward (except for a control group that will not have the feature). All other users whose accounts were created before the deployment date will have the feature turned off, but will be able to go to their preferences to turn it on if they wish to try it out. We will also be embedding an option to turn the help panel off inside the feature. Our team discussed the possibility of turning the help panel on for all users, but wanted to first learn about how new users interact with it, and we want to make sure not to flood help desks with incoming questions. See this Phabricator task for the discussion.
• Namespaces: the help panel will be available in all namespaces. Our team discussed limiting the feature to only the main article namespace because the help documentation linked in the panel is mostly relevant to that namespace. Ultimately, we decided to make it available in all namespaces so that we can learn in which namespaces people are most commonly looking for help -- we know that new editors struggle to edit the Talk namespace. We will also encourage communities to develop documentation relevant to editing other namespaces (particularly the Talk namespace) so that we can make those links available in the appropriate namespaces. See this Phabricator task for the discussion.
• Headers: the help panel will automatically post users' questions to their wiki's help desk, and will automatically generate a header for that post. We have decided that the header will include the title of the page from which the question originates (with a link), as well as a timestamp to distinguish it from questions that may have originated from that some page. Users will be able to decline to include the title in their header, if they consider it sensitive. See this Phabricator task for the discussion. Our preference is to number the posts instead of including a long and redundant timestamp, but that will require additional engineering work for a future version.

Wording for initial version

The wording for the help panel is complete, and the current state can be seen in the interactive mockups. Some of the most important things we have been considering in the wording are:

• How to make it clear that by asking a question in the help panel, users will be posting in public?
• How to encourage users to add their email address, since that is the best way for them to find out when they get a response to their question?
• How to give users the option of excluding the title of the page they're editing from the automatically-generated header in the help desk, if they want to keep that private?
• How to correctly set expectations about how quickly a user can expect a help desk response?
• What to include in the automatically-generated header in the help desk?

Future versions

In determining the initial version of the help panel, many ideas for features were set aside to be evaluated later on. Many of them are visible in the initial mockups here. The following is a list of capabilities that are either being built for the help panel, or may be built in the future:

• Nearer term
  •  - Search: the ability to search help pages. Users found this option appealing during testing, and it was built in T209301 and deployed on 2019-02-14.

    

    Wikipedia help panel feature in Korean Wikipedia, with the "search" functionality in use.

  •  - More contexts: since help desks have not yet been overwhelmed with incoming questions, we are thinking of exposing the help panel to newcomers in contexts beyond editing -- perhaps while they are viewing pages in the Help or Wikipedia namespaces. We thought of this because the EditorJourney analysis showed that large portions of newcomers viewed pages in those namespaces during their first 24 hours. Currently being built in T215664.
  • Greater affordance: so far, it looks like a minority of users who see the help panel button actually open it up. We could attempt to make it more visually obvious that it is a recommended place to start.
  • - Vary links with context: the help panel contains five links to help pages, and about half of users that see them click on one. Different links are probably valuable in different contexts. For instance, there might be some links for visual editor and some for wikitext. Or some that are helpful on a talk page and some on an article page. This is being worked on in T211117.
• Longer term
  • Live chat: allowing newcomers to immediately chat with experienced editors right from the help panel. This would be a major project with many challenges. See this page for our notes so far.
  • Suggested answers: many questions that users asked may have already been answered before, and we might be able to provide suggested answers as users type. This would be relevant once many more questions have been asked. Ticketed in T209327.
  • Feedback on responses: newcomers may find the responses to their questions more or less satisfactory. By allowing newcomers to rate their responses, we may increase higher quality responses, and make it possible to surface the best ones to future newcomers. Ticketed in T209332.
  • Marking question context: newcomers may have questions related to very specific elements of the editing experience. This idea would allow them to indicate exactly where on the page their issue is occurring. Ticketed in T209328.
  • Templated replies: research shows us that the tone of a reply to a newcomer can make a difference in whether they edit again. We may be able to help people answering questions to format their replies in a nurturing way. Ticketed in T209331.
  • Anonymous editors: though our team is focused on registered newcomers, it is possible to make the help panel present for anonymous editors, though it will be more difficult for them to find out that they have received a response. Ticketed in T209325.

User testing for help panel

During the week of November 26, 2018, we used usertesting.com to conduct eight tests of the help panel interactive prototype with internet users unaffiliated with the Wikimedia movement. Four tests were done on desktop and four were done on mobile. In these tests, respondents are compensated for trying out the mockups, speaking aloud on what they observe, and answering questions about the experience. As our team's designer described on the Phabricator task, the goals of this testing were:

1. Gauge the discoverability of the help pane call to action and help pane
2. Identify improvements to the usability of the help pane:
   • Do users understand they can edit and use help links at the same time?
   • Are users able to successfully follow the steps to post a question?
   • Is there anything users that would like to be able to do from the help pane that they are missing?
3. Gauge user reactions to the help pane and expectations of how their questions will be answered.

Summary of findings

• All users found the help panel contents to be very useful in getting help for editing.
• No one had issues with having the article title included in the question.
• Most participants would post a question to the help desk as a last resort after trying to help themselves through clicking on links, with two people saying they would likely not post at all as they would want a more immediate response.
• One user said they would more likely just Google the editing help over the options shown.
• Whilst it was understood that the Help desk would be answered by volunteers, many participants wanted more reiteration and clarity of expected wait times.
• Clear layout of options, shown in the order in which almost participants said they would seek for help.
• Discoverability of the help panel call-to-action may be reduced in the case when the user is using the Visual Editor or wikitext2017 editor because those editors have a competing “?” icon.
• One user suggested having an example question in hint text.
• One user wanted more information shown about when their question was viewed and by how many people.
• Two users mentioned the help recommendations shown on the Help Desk seemed disjointed from what was shown in the help panel.

Recommendations (not all of these will be implemented)

• Prioritize the Searching help task inside the help panel.
• Provide clearer times on both the review and confirmation screens for when a newcomer should expect a response from the help desk.
• Add a more specific help question sample in the placeholder/hint text.
• Experiment or A/B test alternative text on the initial “Ask a question” call-to-action.
• Show a first-run tooltip/pulse indicator to highlight the new help panel.
• Add an additional menu item to existing editing toolbar help on Desktop VE or Wikitext2017 which opens the help panel.

Measurement and results

Measurement and experiment plans

High-level questions

These are the main things we want to find out from the help panel.

1. Does the presence of the help panel lead to questions being asked and answered?
2. Does the presence of the help panel increase editor activation (making their first edit)?
3. Does the presence of the help panel increase editor retention (coming back to edit again)?
4. Does the presence of the help panel increase the proportion of constructive edits?

Controlled experiments

See this page for the detailed experiment plan.

In order to understand the help panel’s impact on editor activation and retention, we propose a six month A/B test. During that test, 50% of new registrations on target wikis will have the help panel enabled by default, and 50% will have it disabled. We anticipate that we will be able to detect 10% changes in activation after about one month, and 10% changes in retention after about six months. We are likely to be running other experiments on the target wikis at the same time, for example, testing variants of our welcome survey. Therefore, assignment of users to treatment and control groups for all tests will have to be coordinated so that we ensure the distributions are not biased.

Specific measurements: quantitative

These are specific measurements that help us answer the high-level questions. These are measurements we will be able to do programmatically. See this EventLogging schema for the exact details on the data that is being recorded and stored.

1. What is the context the user is in when they start interacting with the help panel?
2. What percent of users open the help panel?
3. What percent close the help panel without using any part of it?
4. What percent click one or more of the help links?
5. How far do users go in the workflow of asking a question?
6. What percent of users turn the help panel off?
7. What is the average length of questions asked via the help panel?
8. What percent of users click any of the links in the final confirmation page of asking a question?
9. What percent of users who post a question return to view the answer?
10. Does interacting with the help panel alter the probability of abandoning an edit session?
11. What percent of users use the help panel more than once?
12. Are newly registered users with an email address likely to return to view the answer to their question?
13. What percent of newcomers that had no email address add one when asking a question?
14. What percent of newcomers confirm their email address (within 48 hours) after asking a question?
15. What percent of newcomers ask a question without an email address?
16. What percent of newcomers submit a question without including the title of the page they were editing?
17. How does help panel usage vary with welcome survey responses?

Specific measurements: qualitative

These are specific measurements that help us answer the high-level questions. These are measurements we will need to do manually, by counting and reading the contents of help desks.  Doing these programmatically would be an inordinate amount of work.

1. What percent of questions get answered?
2. How quickly do questions get answered?
3. How often do newcomers edit the question they asked?
4. What kind of questions do people ask and which ones are more likely to get answered?

Usage counts

After about two months of being deployed to half of new accounts in Czech and Korean Wikipedias, and two weeks of being deployed to half of new accounts in Vietnamese Wikipedia, these are the numbers (as of 2019-03-28):

• 3,343 users have seen the help panel button
• 566 users have opened up the panel
• 225 users have clicked a link in the panel
• 51 have run searches
• 51 users have submitted a question to the help desk (Czech: 34, Korean: 8, Vietnamese: 9)

Leading indicators

The help panel's experiment plan defines a set of leading indicators that the team evaluates in order to determine whether the feature is behaving as expected, and whether there are any urgent problems that need to be solved.

Using a month of data, we published an initial evaluation of leading indicators here. In summary, the help panel seems to be behaving in a healthy way, with good numbers of newcomers opening and using it. The analysis identified a couple areas for improvement that the team has taken action on:

• Because we wanted to increase the number of newcomers who open the panel, we started showing it in additional contexts on March 4, 2019. Newcomers now see the help panel when reading in the Wikipedia, Help, and User namespaces.
• Because we saw that in Korean Wikipedia very few users ask questions, we built the search feature so that newcomers would have an easier way to find helpful information on their own.

Experiment results

See results above.

Resources

• More on this feature: The help panel contains a link to learn "More about this feature". That link will lead to this page, which will hopefully answer user questions about how the feature works. Feel free to translate it into your language!
• Best practices: Because many communities don't have a lot of experience concerning help desks, the WMF Community Relations Team has assembled on a page with some best practices that can help communities have more successful collaborations with newcomers. Please feel free to translate that page into your language!