User:AHollender (WMF)/sandbox/In-page navigation/Research

The main goal of this project is to enable readers to navigate through different sections of pages more quickly so that they can find the content they want, in less time and with less scrolling. To this end four designs were prototyped. Starting on May 11th, 2018 we conducted qualitative user testing on two of the prototypes in an attempt to answer our three main research questions:

RQ1 - Are the proposed navigation features (e.g. sticky headers) discoverable (without any kind of guidance or onboarding feature)?
RQ2 - Once discovered, are the proposed navigation features intuitive and easy to learn? (i.e. are they usable without confusion?)
RQ3 - Are the proposed navigation features useful? Once learned, do readers use the features in order to navigate around the page?

Relevant past research includes: Which parts of an article do readers read, and Collapsed vs uncollapsed section view on mobile web

Methodology & participants

Remote, unmoderated user tests conducted through usertesting.com
Users looked at an HTML prototype
Users are non-experts
2 test groups
Testing script

	Group 1	Group 2
Concept tested	#3, sticky section headers	$5, sticky section headers plus TOC
Group details	- 6 participants - US (2), India (1), Australia (1), UK (1), Canada (1) - Female & male - Ages 23–57	- 6 users - US (3), India (1), UK (1), Canada (1) - Female & male - Ages 24–44
Image of concept	(Concept 3) sticky section headers	(Concept 5) sticky section headers + toc

Results

Top level findings are listed here. For more detail please see PDF below.

RQ1 - Are the proposed navigation features (e.g. sticky headers) discoverable (without any kind of guidance or onboarding feature)?

Group 1: ✓ Yes – 4 of 6 testers discovered the feature within the first few tasks of the test. While this is not statistically significant, I find it pretty convincing how quickly users found the feature, and wonder, given a longer test, if the others would’ve discovered it as well.
Group 2: ~ Sort’ve – 2 of 6 testers discovered the feature, with one of them discovering it on the second task. I think it’s fair to say that the feature is discoverable, however perhaps not at a level that we might consider satisfactory.

RQ2 - Once discovered, are the proposed navigation features intuitive and easy to learn? (i.e. are they usable without confusion?)

Group 1: ✓ Yes – All users were intuitively able to use the feature.
Group 2: ✓ Yes – All users were intuitively able to use the feature.

RQ3 - Are the proposed navigation features useful? Once learned, do readers use the features in order to navigate around the page?

Group 1: ✓ Yes – This grade is based on the fact that all users continued to use the feature after discovering it, combined with positive sentiment comments they made.
Group 2: ✓ Yes – This grade is based on the fact that all users continued to use the feature after discovering it, combined with positive sentiment comments they made.

Conclusion

asdasdasd