I was going to post this under Individual Feedback, but it was getting too long and detailed.
- Why should newcomers find talk pages (in the broad sense) and which ones they should be finding? English Wikipedia, for example, has a lot of venues: Teahouse, Helpdesk, per-article Talk, Village Pump, numerous Wikiprojects... Plus there's non-talk help content: help pages, Community Portal, guidelines and wiki-essays etc. So many places that it's hard to know where to turn, and not just for newbies. New user tests covered article discussion on English Wikipedia using the desktop site with Vector skin, so I'll focus on that for a start.
- Yes there are two links on the page presented as "Talk". (French and German WP have discussion and diskussion in both places: are there any languages that have different words in the two places? I notice on this MW page I see page-"Discussion" and user-"Talk". Same on Commons in English, though on Commons in Italian the labels are discussione/discussioni.) In Firefox, if I hover my mouse over one, the tool tip says "Your talk page [Alt+Shift+n]", and the other says "Discussion about the content page [Alt+Shift+t]". I also get URL tips that say "https : //en.wikipedia.org/wiki/Talk:PageName" and "...wiki/User_talk:MyName". Maybe I'm a bit 'special' because I actively seek out these extra clues when exploring somewhere new? Once you get the concept that all the small links to the right of your name apply to you, it's much more convenient than having them hidden behind a drop-down. Understanding why a not-logged-in user still has Talk and Contribs is a bit more complicated, though the tool-tips do say "from this IP address". Those affordances would be harder to activate on a touch-screen tablet.
- The content and talk tabs are right next to each other. I can't think how "the connection between article content and discussions" could be any more visible. Given that MW is page-based and the article-talk happens on a separate page from the article-discussion, the pair of adjacent tabs embodies the mental model that people need to develop to deal with the site.
- If "discuss this page" is an action you do, then it would belong on the right with the other actions. But "page discussion" is a place you go. The test procedure says users were prompted to "talk about how they would find the discussion area", which is consistent with the place-you-go approach. Possibly they just don't see the pair of selectors on the left. If we move the Talk tab to the right, how are people going to understand that Edit source and View history now apply to the talk page not the content page? (Having said that, the grouping in Monobook is more subtle. Surely the change of tab placement was seen as an improvement when Vector replaced Monobook?)
- Vector is confusing in that you have two groups of tab-like selectors and each group has an active tab. It's an elegant use of space but goes against expectations of what a tab does. The normal paradigm would be to have Read|Edit|History stacked underneath Article|Discussion. This could work in Vector if we pushed those tabs up higher (i.e. tabs on left, user-account links and search on right). It would also be interesting to test usability with the whole head section pinned to the top, but that could be less desirable on small screens than big ones.
- It would be really interesting to have the user test repeated with Timeless skin in a wide-enough window. Article, Talk, View, Edit, etc. are more visible, are separated from the user drop-down, and are styled with underlines rather than as tabs.
- Then there's Minerva and the "mobile site" where Talk is a button and is at the bottom of the page instead of top. I went for a long time thinking that there was no Talk or History on mobile before realizing.
[Should maybe have posted each dot-point as a separate comment to aid perma-linking?]