XTools/Development

Thanks for contributing to XTools! This guide gives a high-level walkthrough of everything you need to contribute to XTools in a local environment.

Prerequisites

• PHP 7.4+
• Composer
• Node with the version specified by the .nvmrc file
• MySQL 8.0 or higher, or MariaDB 10.5 or higher (the latter is what's used on production)
• A Wikimedia developer account
• Access to Toolforge so that you can connect to the Toolforge wiki replicas

Running a development server

• Clone the repository: git clone https://github.com/x-tools/xtools.git && cd xtools
• Install PHP dependencies: composer install
• Run cp .env .env.local and fill in the missing values, using /Configuration as a guide.
• npm install to install Node dependencies
• Assets are generated with Symfony Encore. If you're working on JavaScript, CSS, or adding images, compile the assets with: npm run watch.
  • Before submitting your pull request, npm run build to build the production assets.
• Launch Symfony's built-in server: symfony serve
• Assuming the configuration was done correctly and as desired, you should be up and running at http://localhost:8000

The Simple Counter is the simplest tool and should work as soon as you set up XTools. Test it by going to http://localhost:8000/sc and put in Jimbo Wales as the Username and en.wikipedia.org as the wiki. After submitting you should quickly get results.

The development server does not cache application data; any changes you make are visible after refreshing the page. However when you modify things in .env or otherwise aren't seeing changes you're making, you may need to clear the cache with symfony console cache:clear.

The logs are in var/logs/dev.log. If things are acting up unexpectedly, you might try clearing the cache or restarting the server.

Developing against Toolforge replicas

First make sure you have a Wikimedia developer account and access to Toolforge. Then you can set up the necessary tunnels and a shell session with:

ssh -L 4711:s1.web.db.svc.wikimedia.cloud:3306 -L 4712:s2.web.db.svc.wikimedia.cloud:3306 -L 4713:s3.web.db.svc.wikimedia.cloud:3306 -L 4714:s4.web.db.svc.wikimedia.cloud:3306 -L 4715:s5.web.db.svc.wikimedia.cloud:3306 -L 4716:s6.web.db.svc.wikimedia.cloud:3306 -L 4717:s7.web.db.svc.wikimedia.cloud:3306 -L 4718:s8.web.db.svc.wikimedia.cloud:3306 -L 4720:tools.db.svc.eqiad.wmflabs:3306 username@login.toolforge.org

Replace username with your Toolforge Unix shell username. Note our tunnel has to connect to each database slice. The ports used here should match up with the corresponding options in your .env, like so:

DATABASE_REPLICA_USER=<your-username>
DATABASE_REPLICA_PASSWORD=<your-password>
DATABASE_REPLICA_HOST_S1=127.0.0.1
DATABASE_REPLICA_PORT_S1=4711
DATABASE_REPLICA_HOST_S2=127.0.0.1
DATABASE_REPLICA_PORT_S2=4712
DATABASE_REPLICA_HOST_S3=127.0.0.1
DATABASE_REPLICA_PORT_S3=4713
DATABASE_REPLICA_HOST_S4=127.0.0.1
DATABASE_REPLICA_PORT_S4=4714
DATABASE_REPLICA_HOST_S5=127.0.0.1
DATABASE_REPLICA_PORT_S5=4715
DATABASE_REPLICA_HOST_S6=127.0.0.1
DATABASE_REPLICA_PORT_S6=4716
DATABASE_REPLICA_HOST_S7=127.0.0.1
DATABASE_REPLICA_PORT_S7=4717
DATABASE_REPLICA_HOST_S8=127.0.0.1
DATABASE_REPLICA_PORT_S8=4718
DATABASE_TOOLSDB_HOST=127.0.0.1
DATABASE_TOOLSDB_PORT=4720

Change the your-* bits to your own values, which you can find in your replica.my.cnf file in the home directory of your account on Toolforge.

Caching

XTools should probably take advantage of Doctrine's built-in caching mechanism, but it doesn't. Instead it is done in a somewhat manual fashion. This is only done in Repository classes, using this pattern:

public function doSomething($input)
{
    $cacheKey = $this->getCacheKey(func_get_args(), 'component_method');
    if ($this->cache->hasItem($cacheKey)) {
        return $this->cache->getItem($cacheKey)->get();
    }
    $results = 'results of big query';
    return $this->setCache($cacheKey, $results);
}

The cache key can be anything, so long as it is unique to the specific method. A third parameter can be passed to setCache to set the TTL, using the same syntax from the DateInterval class (e.g. P1D is one day, PT1H is one hour).

The above methods are just wrappers around a PSR-6 implementation, intended to reduce the repetition of similar lines of code.

Style guidelines

• It's called "XTools", with two capital letters.
• XTools conforms to PSR2 and Slevomat coding standards; use ./vendor/bin/phpcs -s -p . to check your code.
• HTML routes must begin with the tool name, i.e. EditCounterYearCounts, while API routes begin with either Project, User, or Page.
• Version numbers follow Semantic Versioning guidelines.

Tests

Tests are located in the tests/ directory, and match the src/ directory structure. They are built with PHPUnit. Repositories only handle fetching data and do not need to be tested. Controllers also interact with the database, and while tests are most welcomed for these, they will not run on the CI server (GitHub Actions or Scrutinizer) due to limitations. Instead, put all things that query the replicas behind a app.is_wmf check (example).

There are also tests for linting, phpDoc blocks, and file permissions.

Use composer test to run the full suite, or ./vendor/bin/phpunit tests/ to run just the unit tests.

Releases

Releases are made by tagging commits in the master branch. Before tagging a new release:

• Update the APP_VERSION in the .env file
• Check the copyright year in README.md
• Update RELEASE_NOTES.md with any notable new information for the end user.

Then tag the release (follow the Semantic Versioning guidelines, and annotate the tag with the above release notes) and push it to GitHub.

Lastly, update the version and updated parameters at XTools.

Glossary

The following terms are used throughout the XTools codebase. These words may have different meanings in other MediaWiki-related software.

Anon / anonymous

Internal term referring to both temporary accounts and IPs. IP-specific methods will have "IP" in the name.

ArticleInfo

The old internal name for the Page History tool.

Length

In the context of edits, this is the size of the edit in bytes.

Model

A class holding the business logic for a tool.

Registered

Refers to users, edits or pages created by permanent registered accounts.

Repository

A class responsible for communication with the database, API, or external service.

Template

Refers to either the view (a Twig template), or the Template namespace.

Unregistered

Refers to users, edits or pages created by temporary accounts and IPs.

Additional help

• Email: tools.xtools @ toolforge.org
• IRC: ##wikimedia-xtools ^connect
• MediaWiki talk page: Talk:XTools