Jump to content

Extension:QuickInstantCommons

Issue tracker: #MediaWiki-extensions-QuickInstantCommons
From mediawiki.org
Translate this page
Languages:
MediaWiki extensions manual
QuickInstantCommons
Release status: stable
Implementation File repository
Description Provides a performance optimized version of InstantCommons
Author(s) Brian Wolff (Bawolfftalk)
Latest version 1.5
MediaWiki >= 1.42
PHP >= 7.4.3
License GNU General Public License 2.0 or later
Download
Download extension
Git [?]:

README
Parameters
  • $wgUseQuickInstantCommons
  • $wgQuickInstantCommonsPrefetchMaxLimit
  • $wgQuickInstantCommonsPrefetch
Hooks used
Quarterly downloads 86 (Ranked 63rd)
Translate the QuickInstantCommons extension if it is available at translatewiki.net
Issues Open tasks · Report a bug

The QuickInstantCommons extension is a performance-optimized version of $wgUseInstantCommons . It also allows basic thumbnailing of files that are missing a MediaHandler extension on the local wiki. For example, the first page of a PDF will still thumbnail even without Extension:PdfHandler installed, but advanced features like multipage requires that extension to be installed locally.

Performance

Initial testing was done using the page en.wikipedia.org: List of governors general of Canada (revision 1054426240) which has 85 images. Testing was done with caching disabled and an estimated TTFB for the API on an already setup connection of ~250ms.

It took 1,038.18 seconds with MediaWiki core's InstantCommons, 18.50 seconds with this extension with prefetching disabled, and 1.10 seconds with this extension with prefetching enabled.

Installation

  • Download and move the extracted QuickInstantCommons folder to your extensions/ directory.
    Developers and code contributors should install the extension from Git instead, using:cd extensions/
    git clone https://gerrit.wikimedia.org/r/mediawiki/extensions/QuickInstantCommons
  • Add the following code at the bottom of your LocalSettings.php file: 
    wfLoadExtension( 'QuickInstantCommons' );
  • Yes Done – Navigate to Special:Version on your wiki to verify that the extension is successfully installed.
If using this extension, please disable $wgUseInstantCommons as this extension replaces it. If you really need to use both at the same time, use the settings in the Advanced Configuration section.

Configuration

If you were previously using $wgUseInstantCommons, no configuration is needed beyond wfLoadExtension-ing this extension; configuration is only required for advanced usage.

$wgUseQuickInstantCommons
Automatically set up $wgForeignFileRepos to use this extension (defaults true). Set to false if you need to customize something.
$wgQuickInstantCommonsPrefetch
Enable prefetching images based on what's in DB (default: true). Most of the time this is a major performance boost, but its possible that occasionally this might cause work to be done that isn't necessary.
$wgQuickInstantCommonsPrefetchMaxLimit
Max number of images to prefetch if prefetching is enabled (default: 1,000). Could hit OOM issues if too high.

Advanced Configuration

You can directly configure $wgForeignFileRepos if you want. For example, if you want to use this extension with a wiki that is not Commons. It is also required if you want to use this at the same time as normal instant commons (not reccomended). Example: 

wfLoadExtension( 'QuickInstantCommons' );
$wgUseQuickInstantCommons = false;
$wgForeignFileRepos[] = [
	'class' => '\MediaWiki\Extension\QuickInstantCommons\Repo',
	'name' => 'commonswiki', // Must be a distinct name
	'directory' => $wgUploadDirectory, // FileBackend needs some value here.
	'apibase' => 'https://commons.wikimedia.org/w/api.php',
	'hashLevels' => 2, // Important this matches foreign repo if 404 transform enabled.
	'thumbUrl' => 'https://upload.wikimedia.org/wikipedia/commons/thumb', // Set to false to auto-detect
	'fetchDescription' => true, // Optional
	'descriptionCacheExpiry' => 43200, // 12 hours, optional (values are seconds). This cache is not adaptive.
	'transformVia404' => true, // Whether foreign repo supports 404 transform. Much faster if supported
	'abbrvThreshold' => 160, // must match what foreign repo uses if 404 transform enabled. Default is 255. Wikimedia uses 160.
	'apiMetadataExpiry' => 60*60*24, // Max time metadata is cached for. Recently changed items are cached for less
	'disabledMediaHandlers' => [TiffHandler::class] // media handler extensions to not use. For 404 handling its important that the local media handler extensions match the foreign ones.
];

apiMetadataExpiry is probably the most important value from a performance perspective (other then $wgQuickInstantCommonsPrefetch). If you feel things are slow, try increasing it. If a new version of a file is uploaded, it might not show up properly until after metadata expires. However you can force an expiry by purging the file description page on the local wiki. If you want to maximize performance, an appropriate value might be 60*60*24*30 = 2,592,000 (1 month)

Unlike MediaWiki core's ForeignAPIRepo, $wgResponsiveImages should not impact performance when transformVia404 is enabled, so it is recommended that you keep it turned on.
If the foreign repo also has a foreign repo (for example: en.wikipedia.org), then there won't be a single valid thumbUrl. Instead set it to false for the extension to try and auto-detect.

Differences between QuickInstantCommons and ForeignApiFileRepo

  • Caches curl handle
  • Uses HTTP/2 multiplexing (Major performance boost)
  • ?action=purge of File pages purges image metadata
  • Does not support local thumbnail caching
  • Automatically generate thumbnail URLs locally if foreign repository supports 404 handling (Major performance boost, especially when responsive images are enabled)
  • Prefetch expected images based on imagelinks table (Major performance boost)
  • Allows limited thumbnailing of files that are missing handler extension (e.g. Still show thumbnail of first page of a PDF even if you don't have Extension:PdfHandler installed).
  • Adaptive TTL support that actually works (If the foreign image was recently edited, cache for shorter time, so that if its vandalism the revert will show up faster).
  • Configurable metadata caching time, with better defaults than core.
  • When using a foreign repo that itself has foreign repos (For example, how en.wikipedia.org includes commons.wikimedia.org), image page descriptions will now work correctly.
  • File redirects for files that are foreign files on the foreign repo work.
  • Allows disabling media handlers that are incompatible and use fallback behaviour instead.
  • For images that can't use 404 rendering (missing local media handler extension or if transformVia404 false), predictively prefetches the responsive high-dpi thumbnails
  • Reuse HTTP/2 connection for fetching description page text
  • Do not show text of 404 if the description page is missing on foreign wiki. Instead just show it as blank.

Other ideas that haven't been implemented: better adaptive cache for image description pages.

Known issues

  • If you are using Extension:TimedMediaHandler, you need to use at least MediaWiki 1.38 for this extension to work with video files.
  • If you get an error about undefined constant and the constant name has CURL in it, please ensure that the php curl extension is using at least version 7.43.0 of curl.

See also

Retrieved from "https://www.mediawiki.org/w/index.php?title=Extension:QuickInstantCommons/en&oldid=6675046"