Wikipedia.org Portal Project

Group:	Discovery
Start:	August 2015
End:	December 2017
Team members:	UX: Jan Drewniak; Analysis: Mikhail Popov
Backlog:	#discovery-portal-sprint
Lead:	Deb Tankersley (product owner)
Updates:	Status updates

The Wikipedia.org Portal is a project within the Discovery Department to update and change the wikipedia.org portal page to better serve Wikipedia visitors.

Our early efforts in this project are to measure how visitors use the portal page, learning about user expectations, and determining how we can serve up better and more relevant content. Once the Discovery Portal team has gathered data from test and have analyzed it in-depth for success, we'll promote updates to the portal page in production.

Rationale

The Wikimedia Discovery Portal Team wants to improve the experience of visitors to the wikipedia.org portal page and help them get to the information they are looking for as fast as possible. The portal page gets around 14 million daily page views daily, which is (very roughly) 3% of our total pageviews.

To better serve this large audience, we want to make sure that this page is easy to understand and use. We want to increase the usability of the search text box, so more people will run a search query and get to their preferred article faster. We want to decrease the number of people who leave the page without taking any action. We also want to better understand what people expect from this page so we can address their needs. In the future, we want to serve language, location and time specific content on this page too.

Goals

Decrease bounce rate on the wikipedia.org portal page
Increase amount of searches and clickthroughs on the search results
Increase clickthrough to the top 10 language wikis
Enhance and promote discovery of languages that are not in the top 10
Increase clickthrough to sister (other) Wikimedia projects

Current Quarter Goals

See Wikimedia Engineering's current quarter goals for the team (for 2016-17 Q1):

Current quarter goals:

make the wikipedia.org portal a site for exploring new and open content on Wikimedia sites.

Upcoming quarter goals (2016-17 Q2):

for the next few months, the Portal team will take a hiatus to focus on other projects within the Discovery department. We will continue to update the wikipedia.org page with article stats and translations along with small bug fixes and minor enhancements. If there is something that needs to be addressed, please add it to our backlog on our sprint board. We're planning on resuming work on the portal in late Q3 and Q4 (March - July 2017).

Upcoming A/B Testing

Wikipedia.org Portal A/B Testing Page.

We have several upcoming A/B tests planned for the near future. They are documented at Wikipedia.org Portal A/B testing.

Note: The text and mocks that were previously displayed on this page have been moved and updated in the page linked above. This is to avoid any confusion of having mocks in several places and not knowing which is more correct or up-to-date.

A/B Testing: January - March 2016

The Discovery Portal team performed a test En:A/B testing on Wikipedia.org between January 13 - 20, 2016 to measure if the proposed improvements are beneficial to our users (Phabricator ticket). The findings of these tests can be found on Commons. This was an A/B test to determine the impact of various user experience improvements to the Wikipedia portal (www.wikipedia.org) aimed at increasing the rate at which users click-through to our other Wiki projects. Specifically, we looked at increasing the size and prominence of the search box while pairing images and Wikidata metadata with each search result (see the fourth visual mockup below). We found evidence that the combination of these elements results in a 1.7% to 5.5% higher clickthrough rate. Put another way, that's roughly an increase of 300,000 to 1.3 million pageviews a day into our projects - a solid improvement. Absent non-JavaScript support (which we are working on), we recommend the improvement be deployed to the production Wikimedia.org portal, tentatively scheduled for March 2016. (Update: this was released to production on March 10, 2016)

Another test was run to detect and re-arrange the language links around the globe, based on detection of the user's browser's preferred language setting while adding a localized "The Free Encyclopedia" phrase under the Wikipedia wordmark (Phabricator ticket). This test started on March 22, 2016 and will run until approximately April 5, 2016 when we will start analyzing the results.

Analysis work: 2016

For more information about how our data analysis team is working on this project please refer to: Research: Portal experiments.

Meeting: Rethinking portal planning 2016-02-03
Analysis: Results of removing inline JavaScript on wikipedia.org portal page
Meeting: Wikipedia Portal and JavaScript Usage - review meeting notes 2016-02-11

Presentation of Analysis: Browsers Geography and JavaScript Support on Wikipedia Portal (PDF)
Analysis: Referrers of the10% of traffic to Wikipedia Portal that is referred by something other than a search engine
Analysis: Quick, one page report comparing page views and traffic to wikimedia.org vs wikipedia.org
Analysis: Assessment of Portal update and its impact on search rate post-deployment
Analysis: Clickthrough rates, section usage, and language preferences of Portal visitors
Analysis: Descriptive text added to sister project links
Presentation of Analysis: Wikipedia.org Portal Survey - 'How did you arrive at Wikipedia.org?' May 2016
Analysis: Wikipedia Portal AB Test of Collapsing links to Wikipedia in other languages
Analysis: Retrospective of the Portal work done to date (Dec 2015 - Aug 2016)
Analysis: Rise of Wikipedia.org Portal Pageviews in June and July 2016 (phab ticket)
Analysis: Determine if new portal page layout caused a decrease in Ukrainian wiki pageviews (can be viewed as html or pdf report)

Portal Updates: January - March 2016

Portal Updates: April - September 2016

Browsers Supported

We will be following the guidelines listed on MediaWiki for Compatibility. For example, this will mean that the JavaScript that we have on the Portal page will be targeted for IE9 and newer, while IE8 and below would have a basic CSS experience.

Portal Deployment

The portals repository is deployed as a submodule of the mediawiki-config repo. This submodule in this repo needs to be updated to reflect the latest commit.

cd /srv/mediawiki-staging && git pull && git submodule update portals

The portals are then deployed by executing the sync-portals script at the root of the submodule.

cd portals/

./sync-portals 'SWAT: [[gerrit:372118|Bumping portals to master (T128546)]]'

Ideas for the future of the Portal

Read about our April 20, 2016 brainstorming session here.
Add mobile app badge icons (community feedback conducted: June 13 through August 12, 2016)
Update portal page design to use a dropdown for all languages beyond the default top ten (community feedback conducted: July 4 through August 12, 2016)

Automation of Portal Updates

Now that we've modernized the Wikipedia.org portal page, we will wrap up the work by automating the statistics and translation updates for the portal. These updates are expected to take place on a weekly basis and will still require a human to approve the updates before it is released into production. There are quite a lot of tasks that need to be completed before this effort is done and the best way to view the progress is by visiting the epic ticket in Phabricator that captures the scope of the work we're doing. This work began in October (Q2 2017/18) and will be finished in December 2017.

See the automation documentation in Github