Manual:PHP unit testing/Code coverage

MediaWiki Flower

Contents

Tools

PHPUnit

This manual explains how to generate code coverage on your local workstation. This can be useful to understand how coverage works and to see progress on your work before submitting your change to Gerrit.

Ensure you have:

PHPUnit installed.
A code coverage driver installed (pcov is recommended, but other drivers like Xdebug can be used).

Run the following script:

# $ vendor/bin/phpunit --coverage-html docs/coverage [files]
# For example:
$ vendor/bin/phpunit --coverage-html docs/coverage tests/phpunit/unit/languages/LanguageTest.php

Viewing coverage reports

Coverage reports for MediaWiki core, extensions, and all of our separate libraries are published regularly on doc.wikimedia.org. (See the tutorial on how to add your extension to the list).

CoverMe provides an alternative view, sorting functions by how often they are called in Wikimedia production, helping identify functions that will benefit the most from additional test coverage.

Coverage scope

When running PHPUnit on a subset of files or sub directory, the tests are much quicker. However, building the coverage report may still take a long time.

By changing filter whitelist in phpunit.xml to match the scope of your test run, you can generate coverage reports much faster. Copy phpunit.xml.dist to phpunit.xml in the root of your repository, then update the whitelist:

For example:

--- a/phpunit.xml.dist
+++ b/phpunit.xml.dist

	<coverage includeUncoveredFiles="false">
		<include>
-			<directory suffix=".php">includes</directory>
-			<directory suffix=".php">languages</directory>
-			<directory suffix=".php">maintenance</directory>
-			<directory suffix=".php">extensions</directory>
-			<directory suffix=".php">skins</directory>
+			<directory suffix=".php">includes/libs</directory>
		</include>
		<exclude>
			<directory suffix=".php">languages/messages</directory>
			<directory suffix=".php">maintenance/benchmarks</directory>
			<directory suffix=".php">extensions/*/tests</directory>
			<directory suffix=".php">skins/*/tests</directory>
		</exclude>
	</coverage>

In case the relevant file(s) are in a large directly (e.g. /includes), you can also specify individual files:

--- a/phpunit.xml.dist
+++ b/phpunit.xml.dist
		<include>
-			<directory suffix=".php">includes/libs</directory>
+			<file>includes/includes/WebRequest.php</file>
		</include>

For extensions, you can use the composer script included with MediaWiki core to do this automatically:

Terminal

$ composer phpunit:coverage-edit -- extensions/YourExtension

PhpStorm

In PhpStorm, you can use the "Run {test} with Coverage" option, and you will see the coverage report directly in the editor. As defined above, make sure to edit the coverage include list in phpunit.xml first, doing so results in the coverage report being generated very quickly.

MediaWiki-Vagrant

$ vagrant ssh
$ cd /vagrant/mediawiki/tests/phpunit/
$ sudo -u www-data php phpunit.php --wiki wiki --coverage-html ../../docs/coverage includes/MessageTest.php

You can view the coverage report at http://dev.wiki.local.wmftest.net:8080/w/docs/coverage/.

Use a shortcut:

$ alias mwpun='sudo -u www-data php phpunit.php --wiki wiki'
$ mwpun --coverage-html ../../docs/coverage includes/MessageTest.php

You may get a permissions error about www-data being unable to write to this directory. In that case, try the following:

$ alias mwpun='sudo -u www-data php phpunit.php --wiki wiki'
$ sudo rm -rf /vagrant/mediawiki/docs/coverage /tmp/coverage
$ mwpun --coverage-html /tmp/coverage includes/MessageTest.php
$ sudo mv /tmp/coverage /vagrant/mediawiki/docs/coverage

mediawiki-docker-dev

you@host:mediawiki-docker-dev$ ./bash
root@container:/var/www/mediawiki# cd tests/phpunit/
root@container:phpunit# php phpunit.php --coverage-html ../../docs/coverage includes/MessageTest.php

View the coverage report at http://default.web.mw.localhost:8080/mediawiki/docs/coverage/.

External links

MediaWiki code coverage at doc.wikimedia.org
Whitelisting Files for Code Coverage (PHPUnit Manual)