Topic on Talk:Growth/Focus on help desk/Flow

It increases edits during a newcomer's first day, but decreases them over the following two weeks.

I support improvements, exploration and experimentation. A project may of course require a few revisions before reaching positive results.

If the team doesn't find a way to reverse the longer term negative impact, can someone confirm that there is an explicit plan for full rollback? This is a general concern to avoid accumulating complexity, cruft, and inconsistency across our wikis when some projects don't work out.

@Alsee -- thanks for following along and reading our notes closely. The help panel indeed has ambiguous results. On the one hand, we see anecdotal stories of users who ask questions, get answers, and then continue to edit. On the other hand, we don't see an impact on activation and retention, and we see this ambiguous impact on edit volume. So one thing that you're pointing out that we don't know is whether the help panel seems to increase or decrease edits over the entire first two weeks. I think that so far our analysis looks at Day 1, finds a positive impact, then looks at Days 2 - 14 and finds a negative impact. We can look into what the overall impact is.

But beyond that clarification, our current thinking is that the help panel may work better as part of a broader "integrated newcomer experience", along with the welcome survey, homepage, and newcomer tasks. In other words, it may not be a specific feature that makes the difference for newcomers, but a general overall upgrading of their experience via several connected features. In fact, for newcomer tasks, we plan (in the January/February range) to evolve the help panel to be the feature that guides newcomers as they complete tasks.

All this is to say that we want to keep the help panel around for now and integrate it with the newcomer tasks experience, and see how that affects its usage. If it is not proving useful in that context, then I think we'll need to revisit the questions you're bringing up -- because I agree that we should not leave up features that are not working.

How does this sound?

Thanx. If you reach more positive results that's of course a good thing, and I apologize if it seemed I was jumping the gun about the initial results. I appreciate your confirmation that we should not leave up features if it turns out they're not working. To be honest, the topic caught my interest because there have been difficult times where teams have resisted their own data when it was unfavorable or counter-intuitive. It's easy for good intentions and big positive goals to feed into confirmation bias. The most crucial metric for any project is our inflow of new users as sustained-activity community members. Any negative data in that area should be taken very seriously.