Talk:Talk pages consultation 2019/New user tests

I don't know how much effect it had any effect on the tests, but there's an ugly problem with the sample talk page. It uses fuzzy dead images instead of live templates. It looks and feels really wrong. On immediate sight none of the "links" looked like links. The text and boxes and box-controls didn't look like a natural part of the page. It didn't look like editable native wikicontent.

Not only were all of the elements immediately dull and fuzzy, but moving the mouse across any of those elements would immediately scream that it's "dead" rather than a live native editable wikipage. Moving the mouse over live content makes the mouse pointer change to different types, it results in underlining flickering on any live links, and the link target-address flicker outside of the page in the browser-managament-zones.

Hi @Alsee -- thank you for taking a close look at the results and test materials. We used these images instead of real templates because of how difficult it would be to make all these templates work well in Test Wikipedia, which does not have the full complement of templates that English Wikipedia has. We definitely wanted them to look authentic, and when I looked at them in multiple browsers, we considered them indistinguishable from real templates. But more importantly, when @DannyH (WMF) and I watched the resulting user tests, none of the five testers showed any sign of thinking the images looked weird or fuzzy, and the test did not ask them to interact with the templates, so they did not notice that the links were not real. Taking that all together, we think that the approach here did not negatively impact the tests. Maybe some browsers render the images differently, but fortunately none of our testers seemed to have experienced that.

@MMiller (WMF) at first I was puzzled how you could fail to notice it, but I believe I have figured it out. Using an image-file makes the result resolution dependent. If the images were captured and displayed in the same browser at the same resolution and zoom level, the result probably had high visual accuracy.

It would probably be worthwhile to build a tool to help copy a page (along with all templates it depends on) from one wiki to another. That would help with your testing, the community would find it helpful, and I know wikis outside the WMF-family have wanted help importing some of our templates along with sub-templates it depends on.

@MMiller (WMF): Perhaps you could use Special:ExpandTemplates (on the English Wikipedia) to remove any template dependencies, and then create all of the redlinked articles? I think this would be better than using the still image.

Thanks for the suggestions, @Alsee and @Jc86035. We'll think about these ideas next time we need to run this sort of test again!

I am just speculating, but I think the default Vector skin may be contributing to difficulty finding Talk and other items at the top of the page. Vector styling literally makes the editor-linkset fade into the background.

I would suggest you see if Monobook makes it easier to find Talk, Edit, and the other links for editors. I'm not sure if there's any recent data, but Monobook seems to get a lot of usage by editors. It would be painful to discover that the switch to Vector had diminished our inflow of people finding the Edit and Talk options.

Another thing, the user tests seem to involve only a single article edit. In real world use new editors typically make multiple edits before any Talk page activity. I expect more editing tend to help them finding the Talk page, and make them more familiar and comfortable when they do arrive at a Talk page to use it. The more edits they've made the more likely they are to recognize that Talk isn't really anything new, that it's just another page to edit. Alsee (talk) 07:04, 13 May 2019 (UTC)

I agree with this; it would be useful to test even with only Monobook, Timeless and Minerva (without making any other changes), since Vector is demonstrably a big factor in how easy or difficult it is to edit. (I think it's also possible that those users who try to edit a talk page first are actually less likely to be retained because of that experience, which would explain the abundance or overabundance of new users who edit articles first.) On the other hand, it could be difficult to get accurate results using the non-default skins because of how much time many Internet users have spent staring at Vector.

@Trizek (WMF), Whatamidoing (WMF): Notifying in case no one from the team has found the new comments.

We are monitoring all the comments, no worries. It is just time consuming to acknowledge that we have seen every posts. :)