Extension:MediaWikiFarm/Internals

Overview

Most of the code is inside the class MediaWikiFarm, except the code used to load the global configuration parameters inside the global scope.

The class MediaWikiFarm is in charge of loading the farm configuration, managing the configuration cache, selectionning the requested farm and wiki, compiling the configuration for a given wiki. The executable file src/main.php is the strict equivalent of LocalSettings.php: it loads the configuration parameters inside the global scope.

There are a lot of various configurations, depending on the mono- or multi-version installation, on the PHP version, on the MediaWiki version, and on the PHP SAPI used (Web or CLI). Various flags and require_once are used to select the right files and functions to load the configuration. The main difficulty is to correctly handle the CLI case, mainly because MediaWiki is mostly designed to have only one wiki per installation.

Testing

MediaWikiFarm is unit tested with PHPUnit. The unit tests check (only for now) the first part of the extension: construction of the object and existence of a given wiki. This is tested in the two installation modes (monoversion and multiversion). The tests successfully run when executed in MediaWiki 1.19 to 1.27 and 1.28alpha, with PHP 5.6 and PHP 7.0.

Executing PHPUnit is like executing any other script in the farm: php extensions/MediaWikiFarm/scripts/mwscript.php --wiki=your.wiki.example.org tests/phpunit/phpunit.php --group MediaWikiFarm. You can add -v or even --debug if you want more details.

The command varies depending on the MediaWiki version tested: (Possibly the first form can be extended to previous versions up to MW 1.17 since the hook UnitTestsList was introduced there, but I didn’t find how to.)

MediaWiki 1.27-1.28alpha:

• php mwscript.php --wiki=mywiki.example.org tests/phpunit/phpunit.php --group MediaWikiFarm

MediaWiki 1.24-1.26:

• php mwscript.php --wiki=mywiki.example.org tests/phpunit/phpunit.php [farmdir]/tests/phpunit

MediaWiki 1.19-1.23:

• Install PHPUnit 3.7.38 somewhere, e.g. download with Git and execute Composer inside
• php mwscript.php --wiki=mywiki.example.org tests/phpunit/phpunit.php --with-phpunitdir [phpunitdir] [farmdir]/tests/phpunit

Note that there are two (one by class) hidden tests automatically added by MediaWiki 1.21 and greater (testMediaWikiTestCaseParentSetupCalled) checking that the setUp function calls its parent. Hence it is normal to see a different number of tests between 1.20 and 1.21.

MediaWikiFarm modes

MediaWikiFarm has many different functioning modes. Ideally testing takes into account this variability to test the realistic execution paths.

MediaWikiFarm modes, by raw order of importance

1. Installation type: monoversion or multiversion
2. Cache: no cache, empty cache, up-to-date cache, expired cache
3. Known environment: yes or no
4. Data source for the MW versions: 'variables' files, 'versions' file
5. Delayed upgrade: 'deployments' file or not
6. Entry point: update.php or not

Incidental MediaWiki characteristics

1. Extension registration: yes or no

Incidental MediaWiki extension characteristics

1. Loading mechanism: Composer-managed, extension registration available, require_once available, both mechanisms available

Notes

• The MW version for a specific wiki can be defined in various files, the diffent name are according to the key names in farms.yml
• The known environment is the internal property MediaWikiFarm::environment: when MWF is started in multiversion mode it does not know at this time the whole environment to compile the whole configuration for the wiki, this is only fully known later; currently this environment is only if the selected MediaWiki version is supporting extension registration or not
• The delayed upgrade is a mechanism to smooth the upgrade process (but it can be deactivated): first you change the MW version, which has no effect, then you run the script update.php, which uses the newly-defined version, and at the end of this script, the new version becomes globally active; this is to avoid a state where MediaWiki runs but update.php has not been executed
• The entry point is a minor mode, only the update.php maintenance script can set MWF in other execution path and only when the delayed upgrade is active

Configuration planes

A configuration parameter can be changed by different partly-intersecting planes:

1. Scope of the configuration file, defined by the key and default key in farms.yml: the broader scope it is, the more different wikis can be impacted

   The key can be fixed (scope=wiki or tag or suffix or all), or the key and default can contain a star (wildcard) to catch many wikis but with some restrictions (e.g. only a suffix)

2. Priority inside a config file: the smaller the scope of the definition is, the higher the priority is

   0=default MW; 1=default value from general default config file; 2=other default value; 3=suffix scope; 4=tag scope; 5=wiki-specific

3. Place in the list of configuration files defined in farms.yml
4. Diff array variable or not: in the first case, only the “diff” is changed (some keys are added); in the second, the whole variable is replaced (possibly it can be an array which is entirely replaced); this plane is completely independant of the others

Strategies

Composer-installed extensions

3 possible strategies:

1. Bold: A script (mwcomposer.php) iterates overs Composer-installed extensions and skins (type: mediawiki-extension or mediawiki-skin), creates as much /vendor/composer directories as there are combinations (= 2^n = \sum_{k=0}^n C(n,k), where n = #extensions), each one with a key (e.g. a hash of its content), then replaces the /vendor/autoload.php by a customised file; this one get the Composer key from MediaWiki (for now when used in multiversion mode, since configuration can be loaded before in this mode) and require_once the right composer subdirectory.
2. Conservative: Similar to the previous scenario, but creates as much MediaWiki directories as there are combinaisons (2^n)
3. Quick bold: Like the bold strategy, but instead of iterating over all combinations (O(2^n)), just iterating over extensions (O(n)), create /vendor/composerKEY for each extensions (inside, it contains all other required extensions and libraries), and read in each composer.json the dependencies. The first step of the iteration remains with all extensions activated, so that Composer handles at the beginning if there are incompatible extensions and we are sure, after this step, there exists a coherent versions set.
   Then, during MediaWiki execution, /vendor/autoload.php loads the required /vendor/composerKEY (possibly multiple such directories) and hence the requested extensions are loaded (possibly a class could be loaded by multiple autoloaders, it’s not grave, and it could be avoided as an improvement). Given the new tendency to autoload classes with Composer (e.g. PSR-4) then activate the extension with wfLoadExtension (PageForms, etc.), we could use the dependency graph obtained from composer.json to correctly sort the various wfLoadExtension functions.
   For implementation, it is probable possible to obtain the dependencies from Composer with a plugin installed in composer.json in mwcomposer.json.

Benefits/drawbacks:

1. Bold:
   • + less disk space used, cleaner (not many MW installations)
   • - bold (more issues expected), the customised binary Composer must be used instead, O(n^2) complexity
2. Conservative:
   • + fully compatible with MediaWiki and Composer, even old versions
   • - more disk space used, less practical to manage MediaWiki
3. Quick bold:
   • + even less disk space used, cleaner (not many MW installations), O(n) complexity
   • - bold (more issues expected), the customised binary Composer must be used instead, it reads composer.json so more maintenance expected, possibly a bit slower at runtime than bold (a per-wiki cache could be created, it could be thought)