Reading/Web/Desktop Improvements/Features/Search
We want for all editors on our projects to have the ability to find the articles they are looking for quickly and efficiently. Our search capabilities are vital to this, yet our current search experience has a few issues and areas for improvement. First, the search is in a non-standard place on the page, making it difficult to find to a user with no previous experience with our sites and easily forgotten for users that do already know its location. Second, while searching, it can take a significant amount of time to find the correct results as we currently do not offer any context on the search results displayed.
Our goal is to improve our search experience in ways that make it easier to find and quicker to use. In particular we want to
- Make our search easier to find by moving it to a more prominent location
- Make our search easier to use by including images and other contextual information for improved scannability of the search results
Feature description and requirements
- The search bar will be moved to a new location in the header of the page.
- The search bar will include an image for the page (if available) as well as a description for the page (if available). Page descriptions will be either local descriptions or wikidata descriptions, depending on the policy of the individual wiki.
Design requirements and guidelines
Location change
New search functionality
User testing
Goals
A user test was performed using usertesting.com which focused on the following:
- Indentifying any major usability issues that we may have overlooked
- Learning more about search behavior by identifying the breakdown in terms of how people submit their search:
- (a) click on a suggested result
- (b) press `enter` on their keyboard
- (c) click the `Search` button
- To see what people think the “Search” button will do
- To see what people think the “Search pages containing X” will do
Study
On usertesting.com we had people do various tasks, some of which required them to search for various articles (without explicitly telling them to use the search box). There were two groups:
Group 1
- 17 people
- People in group 1 searched for //Egypt//, //Cave art//, //Banana//, and //Purple//
Group 2
- 15 people
- People in group 2 searched for //Electricity//, //Banana//, //Willow trees//, //Romeo and Juliet//, and //Purple//
There was a mix of ages and geographies in both groups:
Findings
- 0 people had issues using search
- There were a total of 117 searches submitted:
- 86 were submitted via a suggested result
- 19 were submitted via the `enter` key
- 12 were submitted via the `Search` button
- Regarding what people thought clicking the `Search` button would do:
- 16 of 23 people who answered thought it would take them to the first result
- 7 of 23 people who answered thought it would take them to a list of results
- note: all people assumed that `enter` and the `Search` button do the same thing
- Regarding what people thought clicking the `Search for pages containing Purple` would do:
- 25 of the 25 people who answered thought it would take them to a list of pages that have the word "purple" in them
Notes:
- in the first study people started on the Pancake article and were then asked to go to the article about Egypt. It was ambiguous how they were supposed to get there. 7 out of 10 people tried to find an "Egypt" link within the Pancake article
- one person wondered why the Cave art search result didn't have an image — made me wonder if the articles that don't have images look lower quality next to the ones that do?
- in two cases the search results loaded quite slowly and the people used the `Search` button
- few people used their keyboard to navigate down to suggested search results (maybe 2 or 3)
Quantitative testing
Two A/B tests will be performed on the early adopter wikis:
- A comparison of the old location of the search bar vs the new location of the search bar
- A comparison of the old search widget vs the new search widget
We will be tracking the following metrics:
- Search Sessions initiated
- Search Sessions completed
Our target is a 2.5% increase for each change for an overall 5% increase in search sessions initiated
Summary of A/B test results
Logged-in users AB Test
- We observed an average 28.9% increase in the search sessions initiated across early adopter wikis in the AB test;
Results varied on a per wiki basis, with some observed increases and decreases in search sessions initiated between the two test groups.
- For 7 out of the 12 early adopter wikis, there was an increase in the number of search sessions initiated in the new search widget test group.
- Most increases ranged from about 12% to 22% but search sessions initiated were more than double on Serbian (+108.12%) and Hebrew Wikipedia (+192.37%).
- There were no significant differences in search sessions initiated based on the user's edit count bucket indicating that a user's editing experience is not a factor in the likelihood of them starting a search.
Logged-Out Users Pre and Post Deployment Analysis
- We observed an 8% overall increase (13.4% median increase across early adopter wikis) in search sessions initiated by logged-out users on the early adopter wikis following deployment of the new search widget.
Details are available in the full report.
Release plan
We began deploying the first iteration of these changes, moving the search bar to its new location, September 2020 to Office Wiki and Test Wiki, followed by our early adopter wikis. We deployed the second iteration of the new search experience (built in vue.js) in March 2021. This iteration added images and content descriptions where applicable to the search results. See our main Features page for more details.
Configuration
The new search functionality allows for the ability to display an image and a local or Wikidata description along with each search result. This is configurable per project. The current configuration is:
| Project | Description | Images |
|---|---|---|
| Wikibooks | No | No |
| Wikimedia Commons | No | No |
| Wikidata | Yes | Yes |
| Wikifunctions | Yes | No |
| Wikinews | No | Yes |
| Wikipedia | Yes | Yes |
| Wikisource | No | No |
| Wikispecies | Yes | Yes |
| Wikiquote | Yes | Yes |
| Wikiversity | No | Yes |
| Wikivoyage | Yes | Yes |
| Wiktionary | No | Yes |