User:JDrewniak (WMF)/notes/Search widgets at Wikimedia

This page contains technical research on the possible improvements to the search on desktop completed by JDrewniak (WMF)

Search widgets at Wikimedia

There are currently (As of October 2019) 6 different search typeahead implementations at the Wikimedia Foundation.

1. searchSuggest
2. OOUI TitleWidget & TitleOptionWidget
3. MobileFrontend SearchOverlay
4. www.wikipedia.org search widget
5. Wikidata typeahead for item statements
6. Wikidata typeahead for site search

How do they differ?

Screenshot	Name	Lives in	Used by	Depends on	Uses API
	searchSuggest	MediaWiki Core searchSuggest.js	Vector Timeless Monobook Cologne Blue Modern	jQuery mw.api	opensearch
	TitleOptionWidget	MediaWiki Core mw.widgets.TitleWidget.js	ContentTranslation VisualEditor Special:Search	jQuery mw.api OOUI	prefixsearch
	SearchOverlay	MobileFrontend SearchOverlay.js, uses WatchstarPageList.js for results template and SearchGateway.js for API request	MobileFrontend & Minerva	jQuery mw.api Mustache	configurable via $wgMFSearchGenerator (prefixsearch on Wikipedia)
	WMTypeAhead	wikimedia portals wm-typeahead.js	www.wikipedia.org page	mw.html	prefixsearch
	Wikidata Typeahead	Wikidata extension	wikidata.org	jQuery UI	prefixsearch
	Wikidata site-search typeahead	Wikidata extension	wikidata.org	jQuery UI	prefixsearch

All widgets contain the same basic functionality: Producing a list of search suggestions when typing a query into a search input. On top of that:

• MobileFrontend search widget adds a watchstar beside each suggestion, but it's the only one that lacks support for keyboard navigation.
• SearchSuggest is the oldest and most bare-bones, as it doesn’t include thumbnails in the suggestions.
• The OOUI-based TitleOptionWidget is versatile in that it has the option to render with or without the thumbnail.
• The portal implementation is the smallest and most self-contained, requiring only 200 lines of JS (including comments).

The major differences between the implementations are in the underlying frameworks used to build them. All widgets except the portal depend on jQuery and mw.Api. The mobile widget depends on the Mustache templating library, and the OOUI widget depends on OOUI.

Factors to consider

Performance

These search widgets, on both mobile and desktop, are loaded on every page. The idea of replacing the old SearchSuggest widget has been around for a while (see https://phabricator.wikimedia.org/T125725) and a new specialized OOUI widget was even created https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/255436/, but the major blocker for deploying the OOUI based widget was the need to load OOUI on every page (~100kb).

On mobile, the biggest dependency outside of jQuery is the Mustache templating library. However, Mustache is only an 8kb library and is used for many other widgets on mobile, such as the main menu.

Design

Having 4 different implementations of the same features inevitably leads to design inconsistency. Those concerns have been expressed in this task: https://phabricator.wikimedia.org/T153417.

Instrumentation

There exists 3 search widget related eventLogging schemas: Schema:Search and Schema:MobileWebSearch (which is based on Schema:Search) and Schema:SearchSatisfaction. Schema:Search is the oldest of the three and has been marked deprecated. Instrumentation code for Schema:MobileWebSearch still exists in the MobileFrontend code base in MobileWebSearchLogger.js and is still functional (though buggy). Schema:MobileWebSearch is therefore tied to the mobile search widget.

Schema:SearchSatisfaction is used by the Search Platform team measure the efficacy of search algorithms. Its methodology is described in this research document. It is currently active and uses data collected from the SearchSuggest widget on desktop. The searchSuggest widget emits events with mw.track that are registered with the SearchSatisfaction schema. Since this schema is active, any new or revised desktop search widget should remain compatible with the searchSatisfaction schema and emit the same events.

The feasibility of porting the mobile implementation to desktop

Because of the underlying framework dependencies, neither implementation can be ported from Vector to Minerva & vice versa without incurring a performance penalty. Porting the OOUI widget to mobile would mean loading the OOUI library on every page (~100kb) and porting the mobile implementation to desktop would mean loading Mustache on every page (~8kb). If the Mustache dependency could be dropped from the mobile implementation, porting it over to desktop could be feasible. We would however, still have to add keyboard navigation support to the mobile widget as well as compatibility with the searchSatisfaction schema.

The feasibility of creating a new widget

The search suggestion feature is well suited for newer Javascript frameworks that rely on unidirectional data-flow and explicit state management. A search widget is fairly self-contained in that it doesn't interact or depend on other UI elements on page. With a React-like framework, the core behaviour of a search typeahead can be modelled as 1.) responding to input events 2.) Making a network request 3.) Updating a template automatically based on the network response.

This behaviour can now be achieved much easily than in the past. Using a Codepen with existing HTML/CSS from the design task above, I was able to create a typeahead widget in a few hours using the Preact framework: https://codepen.io/j4n/pen/PooqWQLq