Manual:$wgPoolCounterConf

Translate this page

Performance hacks and limits: $wgPoolCounterConf
Configuration for processing pool control.
Introduced in version:	1.16.0 (r52888)
Removed in version:	Still in use
Allowed values:	(array) or null
Default value:	null
Other settings: Alphabetical \| By function

Details

Configuration for processing pool control, for use in high-traffic wikis.

This configuration array maps pool types to client configurations. A client configuration is an associative array with a class key giving the class name of the client (which should be a subclass of PoolCounter), and any other elements which are passed through to the class constructor. The constructor receives three arguments: the configuration array, the pool type and a queue name. (The pool type is used to identify the type of work. The queue name determines which requests are trying to do the same work. E.g. the pool type could be article rendering, and the queue name the ID of the revision which is being rendered.)

An implementation using Redis (PoolCounterRedis) is included with MediaWiki, and another one using a C daemon (PoolCounter_Client) is available in PoolCounter#Configuration.

Pool types

Pool types that can be used as keys for the $wgPoolCounterConf array. More types can be made available by extensions:

ArticleView: When rendering the contents of a page that it's not in parser cache (including old versions of a page).
GetLocalFileCopy: When retrieving a file of more than 10MB to local path for thumbnailing.
diff: When rendering a diff, if the size of the old and new text is greater than 20000 characters.
FileRenderExpensive: When thumbnailing a file that is considered expensive to thumbnail. The consideration of expensive depends on the file type and image or media dimensions. Only applies to thumbnails rendered directly from thumb.php.
FileRender: When thumbnailing a file. Only applies to thumbnails rendered directly from thumb.php.
ApiParser: When parsing wikitext through the api.
SpecialContributions: When rendering Special:Contributions.

Configuration options shared by all client

timeout: Wait timeout in seconds.
workers: Maximum number of active threads in each pool. Additional workers will wait until a place is freed up.
maxqueue: Maximum number of total threads in each pool. (If this number of workers are already working/waiting, fail instead of wait)
slots: Maximum number of workers working on this task type, regardless of queue name. 0 means unlimited. Max allowed value is 65536. The way the slot limit is enforced is overzealous - this option should be used with caution.

Additional configuration options for the Redis implementation

servers: List of Redis server addresses.
redisConfig: Extra configuration array for RedisConnectionPool.

Usage

To use PoolCounter, subclass the PoolCounterWork class (or use PoolCounterWorkViaCallback which can implement arbitrary functionality with anonymous functions). Three methods/callbacks can be implemented:

doWork(): called when the worker is the first to work on the given task (also when the PoolCounter server is down). Returns the result of the work.
getCachedWork(): called when the worker had to wait on another worker and that worker has finished. Can be used to get a result that has been cached by doWork(). If not implemented or returns false, doWork() will be called.
fallback(): called on wait timeout or when there are too many workers waiting already. Can be used to return some not so good but cheap result (e.g. showing a cached older revision instead of rendering the new one). If not implemented or returns false, error() will be called.
error(): Can be used to show an error.

Example

Example configuration:

$wgPoolCountClientConf = [
    'servers' => [ '127.0.0.1' ],
    'timeout' => 0.5,
    'connect_timeout' => 0.01,
];
$wgPoolCounterConf = [
    'ArticleView' => [
        'class' => MediaWiki\PoolCounter\PoolCounterClient::class,
        'timeout' => 15, // wait timeout in seconds
        'workers' => 5, // maximum number of active threads in each pool
        'maxqueue' => 50, // maximum number of total threads in each pool
    ]
];

Example of using PoolCounter:

// get a thumbnail for $file
$work = new PoolCounterWorkViaCallback( 'FileRender', sha1( $file->getName() ), [
    'doWork' => function () use ( $file, $params ) {
        return $file->transform( $params, File::RENDER_NOW );
    },
    'doCachedWork' => function () use ( $file, $params, $thumbPath ) {
        return $file->getRepo()->fileExists( $thumbPath ) ? $file->transform( $params, File::RENDER_NOW ) : false;
    },
    'error' => function ( Status $status ) {
        return wfMessage( 'generic-pool-error' )->parse() . '<hr>' . $status->getHTML();
    }
] );
$result = $work->execute();

Example using redis

$wgPoolCounterConf = [ 
        'ArticleView' => [
                'class' => 'PoolCounterRedis',
                'timeout' => 300, // wait timeout in seconds
                'workers' => 1, // maximum number of active threads in each pool
                'maxqueue' => 2, // maximum number of total threads in each pool
                'servers' => [ '127.0.0.1:6379' ],
                'redisConfig' => [],
        ],
];