ResourceLoader/Package files

Package files can be used with a ResourceLoader module to bundle multiple script files and data exports, accessible client-side via require().

To enable this feature in a module, use the 'packageFiles' property of a module descriptor. Traditional modules specify multiple script files with the 'scripts' property, which blindly concatenates them as if they were a single script without ability to separately access them client-side.

Why

CommonJS interop

This feature was originally conceived as a way to allow interoperability with libraries written for CommonJS-based environments (such as Node.js and npm), allowing these to be compatible with ResourceLoader out of the box. Libraries that contain multiple files that require() each other, can be registered as a ResourceLoader package module without any source code changes.

Private export and import

The ability to export values from individual files and import them from another module, has also proved useful for modernising JavaScript code in MediaWiki, by no longer needing to attach all classes to a public and global object.

In QUnit tests you can access private files from your dependencies as require('ext.example.foo/Bar.js'). Since MediaWiki 1.41 • change 776352

Data and config bundling

The ability for a module to have multiple "files" that remain individually addressable, has also opened the door to support bundling of JSON files and virtual data exports from PHP (as JSON). The proposal to embed config variables in ResourceLoader modules, thus ended up being implemented as virtual files in the module bundle.

Before this feature existed, embedding config and data in modules was also possible, but required writing your own subclass of ResourceLoaderFileModule, an obscure technique that most developers didn't know about or felt uncomfortable using. Most instances of this technique in MediaWiki core have now been ported to use a virtual file instead.

A more common technique for exporting values from PHP was to export them with $out->addJsConfigVars()^[1] or through the ResourceLoaderGetConfigVars hook, as mw.config keys. This is problematic for performance because:

Startup module config vars were exported on all page views for all users (wastes bandwidth cost), and need to be parsed and processed before your actual modules can begin to download (delays interaction), and were only cached for a short time (frequent re-download).
OutputPage config vars are in the <head> and block downloading of article text (delays visual rendering), and have to be processed before actual modules can start to begin downloading (delays interaction).

Exports bundled using Package files, on the other hand:

Only downloaded if they are needed, together with the code (saves bandwidth).
Only downloaded when they are needed (avoids rendering or interaction delays).

Benefit from longer caching times (only re-downloaded if they change, as part of the module).

How it works

MediaWiki version:

≥ 1.33

A package file can either be JavaScript code (script), or a JSON blob (data). A file's type is inferred based on its extension (.js or .json). Files can be real files from disk, or dynamic files generated by code. Dynamic files are commonly used to export the values of configuration settings or other data from the server, and in those cases they're typically named config.json or data.json.

Every module has one main file. This is the first file listed, unless another file is explicitly designated as the main file. When the module loads, only the code in the main file is executed. The code in the other files is not executed unless and until it is invoked using require().

To invoke a non-main file, use require( './foo.js' ). The path must start with ./ or ../ to indicate that it's a file in the same module (as opposed to the name of another module), and the path must be relative to the current file. The file suffix .js is required. The file may assign a value to module.exports, and this value will be returned by the require( … ) function call.

The main file may also assign a value to module.exports, defining a public interface of the module. Code in other modules may access this value by using require( 'module' ) (without ./ or ../ and .js). The module must be loaded first, as a dependency or with mw.loader.using.

Gadgets

Gadgets also support package files. For details refer to Package Gadgets and Extension:Gadgets#Options. Since MediaWiki 1.38

Virtual files in traditional modules

You can also include callback-generated code or data in a traditional modules. Since MediaWiki 1.41 • change 905155

Example:

{
    "mediawiki.example": {
    "scripts": [
    	"mediawiki.example.js",
    	"mediawiki.example.numbers.js",
    	{
    		"name": "mediawiki.example.data.js",
    		"callback": "Example::getDataJs"
    	}
    ]
}
/*
use RL\Context;
use Config;

public static function getDataJs( Context $context, Config $config ) {
		$data = self::getDataArray();
		return 'mw.example.setData('
		. $context->encodeJson( $data )
		. ');';
}
*/

Module definition

Files can be specified in the 'packageFiles' property in the following ways:

Static files

Static file from the filesystem

"packageFiles": [
  "foo/bar.js"
]

The name under which it is made available to require() can be overridden if needed, using a name alias:

"packageFiles": [
  { "name": "blah.js", "file": "foo/bar.js" }
]

Main module file

By default the first JavaScript file in the packageFiles array will be considered the module's main file (or "entry point").

To set this explicitly instead, use the main attribute. This indicates which of the files executes first and has the control to import other files:

"packageFiles": [
  { "name": "bar.js", "main": true }
]

Dynamic source file

The callback option can be used to dynamically decide which file to relate to an exported name. Read more about what callbacks can do in the sections further down.

"packageFiles": [
    {
      "name": "foo.js",
      "callback": "MyExtensionHooks::getFooFile"
    }
]

function getFooFile( MediaWiki\ResourceLoader\Context $context, Config $config ): MediaWiki\ResourceLoader\FilePath {
    $file = $config->get( 'FooSpecialEnabled' ) ? 'foo-special.js' : 'foo.js';
    return new MediaWiki\ResourceLoader\FilePath( $file );
}

Virtual files

Custom content

Define a virtual JavaScript file, with the specified contents:

"packageFiles": [
    {
      "name": "bar.js",
      "content": "console.log( 'Hello world' );"
    }
]

Define a virtual JSON file, whose contents will serialised as JSON. For example the below would export {"hello": "world"}:

"packageFiles": [
    {
      "name": "bar.json",
      "content": { "hello": "world" }
    }
]

The same works from PHP as well (such as in MediaWiki core's Resources.php file, or from an extension hook):

'packageFiles' => [
    [
    	'name' => 'blar.json',
    	'content' => [ 'hello' => 'world' ],
	],
]

Generated content

Define a virtual file whose contents is generated by a callback. For JS files, the callback should return a string. For JSON files, it can return anything that's JSON-serializable (typically an associative array).

"packageFiles": [
    {
      "name": "blah.json",
      "callback": "MyExtensionHooks::generateBlah"
    }
]

/** @return array|string|int|bool Data for JSON */
function generateBlah( MediaWiki\ResourceLoader\Context $context, Config $config ) {
    return /* ... */;
}

The callback is executed in the context of a load.php request and cached as part of that module, so it can't know which user is logged-in or which page is being viewed. Instead, the result is computed once and re-used across different users and pages. If you did attempt to access a RequestContext or User in your callback, it would likely return Error: Sessions are disabled for this entry point .

The MediaWiki\ResourceLoader\Context and Config do offer information that you can vary by, such as by site configuration, current skin, and current interface language.

The callback takes an optional $param, which is set to the value of a "callbackParam" key specified in packageFiles. This allows callbacks to be re-used for multiple purposes:

"packageFiles": [
    {
      "name": "foo.js",
      "callback": "MyExtensionHooks::generateFoo",
      "callbackParam": [ "A", "B", "C" ]
    }
]

/** @return string JavaScript code */
function generateFoo( MediaWiki\ResourceLoader\Context $context, Config $config, $param ): string {
    return /* ... */;
}

Common pitfall: using i18n messages in generated content callbacks

A common source of errors is trying to use wfMessage() in these callbacks. Trying to do this results in a Sessions are disabled for this entry point error. Instead, you should use the $context object passed in as the first parameter, and replace calls to wfMessage() with $context->msg(). Similarly, you should not use $wgLang, use $context->getLanguage() instead.

function generateBlah( MediaWiki\ResourceLoader\Context $context ) {
    // Wrong, causes an error:
    $numUsersMessage = wfMessage( 'how-many-users' )->numParams( $numberOfUsers )->parse();
    $userList = $wgLang->commaList( $users );
    
    // Right:
    $numUsersMessage = $context->msg( 'how-many-users' )->numParams( $numberOfUsers )->parse();
    $userList = $context->getLanguage()->commaList( $users );
}

Config files

This is a shortcut for the common case of generating JSON that exports one or more MediaWiki configuration variables. The below defines virtual file whose contents are {"LegalTitleChars": "...", "IllegalFileChars": "..."}:

[
	'name' => 'blah.json',
	'config' => [ 'LegalTitleChars', 'IllegalFileChars' ]
]

Note that this syntax uses config setting names as understood by Config::get() (e.g. 'LegalTitleChars'), which are without the wg prefix that global variables use (e.g. 'wgLegalTitleChars').

You can also use aliases to export configuration variables under different names:

[
	'name' => 'blah.json',
	'config' => [ 'naughtyChars' => 'IllegalFileChars' ]
]

will result in {"naughtyChars": "value of $wgIllegalFileChars"}.

If you need to do more advanced manipulation of config variables, use a callback as described above. In the callback, you can use e.g. $config->get( 'IllegalFileChars' ) to get the value of a config setting.

Base path

Most modules using Package files set 'localBasePath' to the common directory prefix of the files. This is done for convenience (not having to write resources/src/whatever/ over and over), and to make dynamic files easier to deal with. Without a base path, a dynamic file called config.json would have to be accessed using require( '../../../config.json' ), or it would have to be named resources/src/whatever/config.json so it could be accessed using require( './config.json' ). With a base path, you get the best of both worlds.

Note that, if you set localBasePath, you will also have to set remoteBasePath (for core) or remoteExtPath (for extensions) to match.

Incompatibility with `'scripts'`

If a module uses the 'packageFiles' property, it cannot use the 'scripts' property. Defining a module that uses both properties will throw an exception.

Package files are also incompatible with 'languageScripts' and 'skinScripts'. Defining 'languageScripts' won't throw an exception, but the property will be ignored. ('skinScripts' throws an exception just like 'scripts' does.) A way of defining language/skin-specific script files for modules using Package files has not yet been developed (and this blocks porting ResourceLoaderLanguageDataModule).

Example uses in real code

Basic example/illustration

MediaWiki version:

≥ 1.33

Module definition (core)

In Resources.php:

    'mything' => [
        // Make all paths relative to resources/src/mything
        'localBasePath' => "$IP/resources/src/mything",
        'remoteBasePath' => "$wgResourceBasePath/resources/src/mything",
        'packageFiles' => [
            'init.js', // Main file because it's listed first
            'thinglib/index.js',
            'thinglib/formatter.js',
            [ 'name' => 'config.json', 'config' => [ 'UseLongThingFormat' ] ],
            [ 'name' => 'data.json', 'callback' => function ( MediaWiki\ResourceLoader\Context $context, Config $config, array $callbackParams ) {
                $language = Language::factory( $context->getLanguage() );
                return [
                    'monthNames' => $language->getMonthNamesArray();
                ];
            } ],
        ],
    ],

Module definition (extension)

In extension.json:

    "mything": {
        "localBasePath": "modules/mything",
        "remoteExtPath": "MyExtension/modules/mything",
        "packageFiles": [
            "init.js",
            "thinglib/index.js",
            "thinglib/formatter.js",
            {
                "name": "config.json", 
                "config": [ "UseLongThingFormat" ]
            },
            {
                "name": "data.json",
                "callback": "MyExtensionHooks::getMyThingData",
                "callbackParam": { "key1": "value1", "key2": "value2" }
            }
        ]
    }

In MyExtensionHooks.php:

class MyExtensionHooks {
    // ...
    public static function getMyThingData( MediaWiki\ResourceLoader\Context $context, Config $config, array $callbackParams ) {
        $language = Language::factory( $context->getLanguage() );
        return [
            'monthNames' => $language->getMonthNamesArray();
        ];
    }
}

JavaScript

In init.js:

var thinglib = require( './thinglib/index.js' ),
    monthNames = require( './data.json' ).monthNames,
    config = require( './config.json' ),
    formatter = new thinglib.Formatter( { months: monthNames } );

if ( config.UseLongThingFormat ) {
    formatter.format( /* something */ );
} else {
    // do something else
}

In thinglib/index.js:

var thinglib = {
    Formatter: require( './formatter.js' ) // note this path is relative to the file we're in
    /* otherthing: require( './otherthing.js' ) */
    /* etc */
};

module.exports = thinglib;

In thinglib/formatter.js:

function Formatter( config ) {
    // ...
}

Formatter.prototype.format = function ( /* ... */ ) {
    // ...
};

module.exports = Formatter;

You no longer need to wrap script files in a namespace closure.

Debugging

It is recommended to debug package files in production mode rather than debug mode. Thanks to source maps (since MediaWiki 1.41, T47514), stack traces automatically expand to the original files in the browser console. Likewise, the "Sources" or "Debugger" tab of the browser's developer tools allow you to navigate each of the package files, and set break points, for example.

If you're having trouble finding a file or a piece of code using your browser's developer tools, you can use the following hack. Suppose you're looking for thinglib/formatter.js in the module named mything from the example above.

In Chrome:

In the console, evaluate mw.loader.moduleRegistry['mything'].script.files['thinglib/formatter.js']
This prints something like f(require,module), which you can click on.
This will take you to the "Scripts" panel . If the code appears minified, click the {} "pretty print" button at the bottom left of the source panel.

In Firefox:

In the console, evaluate mw.loader.moduleRegistry['mything'].script
This prints something like Object{ files: { "thingblib/formatter.js": js() ↱ } }, which you can expand to click on the ↱ up arrow of the file you're interested in.
This will take you to the "Debugger" panel.

Accessing exports from browser console

The require function is specific and local each module's own scope. From the browser console, you can access the public exports of a module via mw.loader.require('mything').

To read the source of private files within each module for debugging purposes, use the "Sources" or "Debugger" tab of the browser's developer tools, based on the Deugging section above. To inspect the live exported value from a private file from the console, run mw.loader.moduleRegistry['mything'].packageExports['thinglib/formatter.js'].

Footnotes

↑ Note that vars that depend on the request context (e.g. the user or the page title) can't be moved to a package module. They can only be exported with OutputPage::addJsConfigVars().

[1] Note that vars that depend on the request context (e.g. the user or the page title) can't be moved to a package module. They can only be exported with OutputPage::addJsConfigVars().

[1]

Why