Jump to content

Reading/Browser tests/Setting up a browser test bot

From mediawiki.org

This documents the steps required to get a bot running browser tests and posting test results to Gerrit. There is more information on wikitech-l.

Steps

  • Acquire a Gerrit account e.g. Barrythebrowsertestbot
  • Acquire a labs project
  • Create a labs instance with mediawiki
  • ssh into your instance e.g. ssh gather-browser-tests.eqiad.wmflabs
  • Install any extensions you want to test into the mediawiki install. It's likely you'll also need to install the vector skin.
  • Install phantomjs
  • Install Ruby (apt-get install ruby-dev && gem install bundler)
  • Install and configure arcanist for pasting failed test results to Phabricator (see below)
  • Install git review
  • Get barrybot.py from BarryTheBrowserTestBot
  • Register a public key with Gerrit and your labs instance.
  • Create a mediawiki account on your machine for your tests e.g. "User:Mr Selenium"
  • Update ~/.profile with information specific to your machine.
  • Navigate to the extension you want to run browser tests for and set up the browser test environment. Ensure the tests are passing green. Note that some extensions e.g. MobileFrontend require additional setup steps.
  • Run a trial run of the BarryTheBrowserTestBot without the --review flag
  • Set up a background task script and watch it review away.

Note: As of June 2015 this reviews any patches which do not have the browser bot attached to them. You can trigger a new review by removing the browser bot from the patchset.

Install and configure arcanist

Install

For full instructions see https://secure.phabricator.com/book/phabricator/article/arcanist_quick_start/ 

* cd /opt
* git clone https://github.com/phacility/arcanist.git
* cd /opt/arcanist/externals/includes/
* git clone https://github.com/phacility/libphutil.git
  • Edit /etc/profile and add the following line: PATH="$PATH:/opt/:/opt/arcanist/bin/" before export PATH
  • source /etc/profile

Configure

.profile sample code

BROWSER# Make sure you add phantomjs and Barry-the-Browser-Test-Bot to your path
PATH="/srv/Barry-the-Browser-Test-Bot:/srv/phantomjs-2.0.0/bin/:$PATH"
MEDIAWIKI_URL
# setup the variables needed by the browser tests
export MW_SERVER=http://gather-browser-tests.wmflabs.org/
export MW_SCRIPT_PATH=/w
export MEDIAWIKI_URL=http://gather-browser-tests.wmflabs.org/wiki/
export MEDIAWIKI_USER=Mr_Selenium
export MEDIAWIKI_PASSWORD=password
export MEDIAWIKI_API_URL=http://gather-browser-tests.wmflabs.org/w/api.php
export MEDIAWIKI_LOAD_URL=http://gather-browser-tests.wmflabs.org/w/load.php
MEDIAWIKI_LOAD_URL
# headless testing for the win
export BROWSER=phantomjs

Background task sample code

#!/bin/sh
while :
do
  # Do Gather - trigger a review on the result. --project corresponds to the Gerrit project you want to test.
  # --core, --test --dependencies correspond to absolute directories on your machine. --core and --dependencies will be switched to master and updated before launching the browser tests.
  ./barrybot.py --noupdates 1 --review 1 --paste 1 --project mediawiki/extensions/Gather --core /srv/mediawiki/ --test /srv/mediawiki/extensions/Gather/ --dependencies /srv/mediawiki/extensions/MobileFrontend
  # Do MobileFrontend but limit browser tests to those that are tagged @smoke
  ./barrybot.py --review 1 --paste 1 --project mediawiki/extensions/MobileFrontend --core /srv/mediawiki/ --test /srv/mediawiki/extensions/MobileFrontend/ --tag smoke
  # sleep for 30 minutes
  sleep 1800
done

Run this in the background to start reviewing!: 

./myscript.sh &&

Current setup at gather-browser-tests.eqiad.wmflabs

  • The username used for pasting test results to https://phabricator.wikimedia.org is 'Barry-the-Browser-Test-Bot'. To get the password (which you probably won't need) run: cat /opt/barry_phab.txt
  • Arcanist and barry are cloned at /opt/arcanist and /opt/barry
  • The run script is located at /opt/runbarry.sh (which should probably be replaced by a cron job)
Retrieved from "https://www.mediawiki.org/w/index.php?title=Reading/Browser_tests/Setting_up_a_browser_test_bot&oldid=4139192"