Reading/Web/Desktop Improvements/Hureo User Research Report/en

Starting in January 2020, we have been working with Hureo, a user research firm based in India, to perform a user study on how new and casual readers use the desktop interface of Wikipedia. The study was separated in two phases, with the first phase focusing on primarily English readers, and the second phase on bilingual and non-English readers. The goal of the study was to understand the experience of new and casual readers, both in terms of how they feel about the site and how they use the site, focusing in particular on the concepts of trust and welcomeness, as well as on the usability of commonly used features on the site. In addition, we were also interested in uncovering potential areas of improvement in terms of overall and feature-specific usability. The outcome of the first phase of the study was a report detailing their observations from 24 user interviews, which is available below. We will be using these results to inform future feature development for desktop improvements project.

Feedback methodology

Hureo created an interview scripts based on the following study goals:

Welcomeness and readability: What are their initial feelings towards the site? Does it feel welcoming? How do they feel about the reading experience generally? Do they understand what the site is and how it works? Do they understand that they can be involved in the creation of the content/knowledge?
Credibility and trust: what are their impressions of credibility/trustworthiness of the site, and what informs these impressions? How do these impressions compare with other sites they use?

The study was performed using ethnographic interviews focusing on behavior observations with some task based questions. The study was focused on two categories on Wikipedia Users, with 12 participants in each group:

Newcomers: people with little to no familiarity with the desktop interface of Wikipedia. They may or may not have visited Wikipedia on their phone.
Casual readers: people who use Wikipedia occasionally to regularly. They spend little to no time thinking about Wikipedia conceptually and probably don’t have a clear mental model of how Wikipedia works.

Summary of results

Positive

Depth and breadth of information: readers liked that Wikipedia contained in-depth information about many topics, and that they did not have to visit any other websites for additional information.
Images: readers appreciated the inclusion of images within the page.
Table of contents: readers appreciated the table of contents both as a navigation tool as well as as means of gaining an overview of the contents of a given page.
No advertisements: when compared to other websites readers highlighted that they appreciated the lack of advertisements on the page.
Language switching: readers appreciated the ability to switch languages on a page.

Challenges

Too much information: readers found the contents of the page overwhelming, in particular noting the lack of “segregated information”, which made it difficult for them to find what they were looking for.
Main menu/sidebar: readers were unable to understand the purpose of the Menu on the left hand side of the site, noting in particular that they did not understand the items in the menu (e.g. Related changes, Special Pages). They felt that it was not relevant for them.
Article tools: readers did not understand the terminology and concept of most of the article tools (e.g. Talk, Contributions).

Ideas

More images and video: we saw a number of requests for more images in the article. Some readers also wanted to see videos related to their topic embedded in the article.
More segregated information: the readers wanted to easily scan the article to find the specific information. They wished the information to be more segregated.
Ease of navigation: the readers were interested in navigating around the page in an easier way, for example, being able to go back to the Table of Contents or other places within the site without scrolling back to the top.
Easier access to the table of contents: some readers mentioned that they would like to see the table of contents throughout reading, perhaps in the form of tabs on the top or the left side of the page, so that they could see the contents of the entire article while reading a particular section.

Complete results

Following steps

There also was the phase 2 of the Hureo-led user research with another report. This phase of the research focused on readers who do not use English as their primary language online. In addition to focusing on the usability and welcomeness of the current site, we also requested of participants to compare the current site to a prototype of some of our proposed changes as a part of this phase.
The results of this phase allowed us to confirm some of our previous hypotheses on site navigation. Mainly, that some aspects of the navigation are confusing and unhelpful to readers while others, which might otherwise be used frequently, were difficult to find. Additionally, this phase raised awareness on the importance of certain navigational elements, such as the table of contents. We will use the information in this report to iterate on and inform our feature development for the remainder of the desktop improvements project.