@Alsee -- thanks for following along and reading our notes closely. The help panel indeed has ambiguous results. On the one hand, we see anecdotal stories of users who ask questions, get answers, and then continue to edit. On the other hand, we don't see an impact on activation and retention, and we see this ambiguous impact on edit volume. So one thing that you're pointing out that we don't know is whether the help panel seems to increase or decrease edits over the entire first two weeks. I think that so far our analysis looks at Day 1, finds a positive impact, then looks at Days 2 - 14 and finds a negative impact. We can look into what the overall impact is.
But beyond that clarification, our current thinking is that the help panel may work better as part of a broader "integrated newcomer experience", along with the welcome survey, homepage, and newcomer tasks. In other words, it may not be a specific feature that makes the difference for newcomers, but a general overall upgrading of their experience via several connected features. In fact, for newcomer tasks, we plan (in the January/February range) to evolve the help panel to be the feature that guides newcomers as they complete tasks.
All this is to say that we want to keep the help panel around for now and integrate it with the newcomer tasks experience, and see how that affects its usage. If it is not proving useful in that context, then I think we'll need to revisit the questions you're bringing up -- because I agree that we should not leave up features that are not working.
How does this sound?