Extension:LocalS3Repo
 | This extension is currently not actively maintained! Although it may still work, any bug reports or feature requests will more than likely be ignored. |
 Release status: unmaintained |
|
|---|---|
| Implementation | File repository |
| Description | File repo for Amazon S3 system. Simulates a local filesystem where the data is stored on the S3 system. |
| Author(s) | antonydandrea |
| Latest version | 2.0 (2017-07-20) |
| MediaWiki | 1.21 - 1.28 |
| PHP | 5.2 |
| License | GNU General Public License 2.0 or later |
| Download | GitHub: Note: |
The LocalS3Repo extension implements the Amazon S3 as a filerepo. The local model was used as a base, and the wiki will use the S3 instead of the local disk drive for the storage of all uploaded files.
Usage
[edit]Once installed, it will operate automatically. There are no user interface issues. You have to have an account with Amazon S3, explained here.
Download instructions
[edit]- Version by antonydandrea stable for 1.28m may work for 1.21+: repository.
- Version by Cariaso for 1.23+ (note from the author: I'm the author of that, and can warn you that while it mostly works, it's a bit crashy): repository.
- Very old version: LocalS3Repo.zip.
Choose a version, download and unzip it to $IP/extensions/, with folders. The files should be in the $IP/extensions/LocalS3Repo/ directory. Note: $IP stands for the root directory of your MediaWiki installation, the same directory that holds LocalSettings.php.
Installation
[edit]Below are 2 versions of what should be added to the bottom of your LocalSettings.php file. Believe the first one will work from v1.21 onwards as much as the second one, as well as the code it is attached to. This repository should be tagged if not.
/*
Modified to work with 1.25.
Antony D'Andrea - contactme at antonydandrea dot com
This also has a couple of bug fixes on the code, but the config below
along with the bug fixes definitely works in MediaWiki 1.25.
*/
// s3 filesystem repo
$wgUploadDirectory = 'wiki';
$wgUploadS3Bucket = 'YOUR S3 BUCKET';
$wgUploadS3SSL = false; // true if SSL should be used
$wgPublicS3 = true; // true if public, false if authentication should be used
$wgS3BaseUrl = "http".($wgUploadS3SSL?"s":"")."://s3.amazonaws.com/$wgUploadS3Bucket";
//viewing needs a different url from uploading. Uploading doesnt work on the below url and viewing doesnt work on the above one.
$wgS3BaseUrlView = "http".($wgUploadS3SSL?"s":"")."://".$wgUploadS3Bucket.".s3.amazonaws.com";
$wgUploadBaseUrl = "$wgS3BaseUrlView/$wgUploadDirectory";
// leave $wgCloudFrontUrl blank to not render images from CloudFront
$wgCloudFrontUrl = '';//"http".($wgUploadS3SSL?"s":"").'://YOUR_CLOUDFRONT_SUBDOMAIN.cloudfront.net/';
$wgLocalFileRepo = array(
'class' => 'LocalS3Repo',
'name' => 's3',
'directory' => $wgUploadDirectory,
'url' => $wgUploadBaseUrl ? $wgUploadBaseUrl . $wgUploadPath : $wgUploadPath,
'urlbase' => $wgS3BaseUrl ? $wgS3BaseUrl : "",
'hashLevels' => $wgHashedUploadDirectory ? 2 : 0,
'thumbScriptUrl' => $wgThumbnailScriptPath,
'transformVia404' => !$wgGenerateThumbnailOnParse,
'initialCapital' => $wgCapitalLinks,
'deletedDir' => $wgUploadDirectory.'/deleted',
'deletedHashLevels' => $wgFileStore['deleted']['hash'],
'AWS_ACCESS_KEY' => 'YOUR_AWS_ACCESS_KEY',
'AWS_SECRET_KEY' => 'YOUR_AWS_SECRET_KEY',
'AWS_S3_BUCKET' => $wgUploadS3Bucket,
'AWS_S3_PUBLIC' => $wgPublicS3,
'AWS_S3_SSL' => $wgUploadS3SSL,
'cloudFrontUrl' => $wgCloudFrontUrl,
);
require_once("$IP/extensions/LocalS3Repo/LocalS3Repo.php");
/*
Modified to work with 1.21 and CloudFront.
Owen Borseth - owen at borseth dot us
LocalS3Repo modified to work with MediaWiki 1.21, maybe others, and CloudFront CDN. A maintenance script that I used to move my current
files over to S3 has been included; it will probably need to be slightly modified to work for you.
*/
// s3 filesystem repo settings - start
// Modify below with tyour settings and paste it all into your LocalSettings.php file.
// Basically, just modify the values that are in all uppercase and all should be fine.
// $wgUploadDirectory is the directory in your bucket where the image directories and images will be stored.
// If "images" doesn't work for you, change it.
$wgUploadDirectory = 'images';
$wgUploadS3Bucket = 'YOUR S3 BUCKET';
$wgUploadS3SSL = false; // true if SSL should be used
$wgPublicS3 = true; // true if public, false if authentication should be used
$wgS3BaseUrl = "http".($wgUploadS3SSL?"s":"")."://s3.amazonaws.com/$wgUploadS3Bucket";
$wgUploadBaseUrl = "$wgS3BaseUrl/$wgUploadDirectory";
// leave $wgCloudFrontUrl blank to not render images from CloudFront
$wgCloudFrontUrl = "http".($wgUploadS3SSL?"s":"").'://YOUR_CLOUDFRONT_SUBDOMAIN.cloudfront.net/';
$wgLocalFileRepo = array(
'class' => 'LocalS3Repo',
'name' => 's3',
'directory' => $wgUploadDirectory,
'url' => $wgUploadBaseUrl ? $wgUploadBaseUrl . $wgUploadPath : $wgUploadPath,
'urlbase' => $wgS3BaseUrl ? $wgS3BaseUrl : "",
'hashLevels' => $wgHashedUploadDirectory ? 2 : 0,
'thumbScriptUrl' => $wgThumbnailScriptPath,
'transformVia404' => !$wgGenerateThumbnailOnParse,
'initialCapital' => $wgCapitalLinks,
'deletedDir' => $wgUploadDirectory.'/deleted',
'deletedHashLevels' => $wgFileStore['deleted']['hash'],
'AWS_ACCESS_KEY' => 'YOUR_AWS_ACCESS_KEY',
'AWS_SECRET_KEY' => 'YOUR_AWS_SECRET_KEY',
'AWS_S3_BUCKET' => $wgUploadS3Bucket,
'AWS_S3_PUBLIC' => $wgPublicS3,
'AWS_S3_SSL' => $wgUploadS3SSL,
'cloudFrontUrl' => $wgCloudFrontUrl,
);
require_once("$IP/extensions/LocalS3Repo/LocalS3Repo.php");
// s3 filesystem repo settings - end
To transfer your files from an existing wiki:
- Create the bucket and the "folder" (the S3Fox tool for FireFox works well for this)
- Transfer all the files/folders in your existing wiki's image directory to your "folder".
- Set the ACL for the folder (and subfolders, etc) to be readable by everyone, if you do not use authentication
I have tested this with Windows 7 and with Linux, with Mediawiki versions 1.15.4 and 1.16.0beta3. I currently have a wiki running on Linux with this extension. I have tested the funcionality of upload, thumbnails, delete, archive, and rollback. There could be some other functions which some other extensions use which I didn't implement or test, watch out!
This extension isn't by any means perfect, because the files are sent first to the wiki server, and from there they are sent to the S3 system on uploads. On file reading, they come directly from the S3 system, which is correct. I am waiting for a new upload screen which has a progress bar, then I will implement sending the files directly to the S3 system on upload. So for now, the file size limit of your PHP server will be the file size limit. I deal with that on my system by doing it manually, because I have few files larger than the limit.
Note that the S3.php file, from Donovan SchĂśnknecht, is included in the zip file, and requires that curl be installed on your PHP server (check your phpinfo).
Configuration
[edit]- $wgUploadS3Bucket --> The name of the bucket you created for the wiki
- $wgUploadDirectory --> The "folder" under the bucket where you wish the files to be stored
- $wgUploadS3SSL --> true if SSL should be used
- $wgPublicS3 --> true if public S3 file access, false if signature authentication should be used
- 'AWS_ACCESS_KEY' --> Your S3 access key, a long string of characters assigned by Amazon
- 'AWS_SECRET_KEY' --> -Your S3 secret key, a long string of characters assigned by Amazon
See also
[edit]- Extension:AWS - uses Amazon S3 as FileBackend (also stores uploads in Amazon S3).
- Manual:Integration_with_S3 - mounts Amazon S3 as a filesystem (Note: not recommended in production)
- LocalS3Repo2 - a modified version that should work with version 1.21, maybe others, and Cloudfront
- LocalS3Repo2Fork - A fork of the above that works with 1.25 and is awaiting a pull request to be accepted in the above one.
Note
[edit]- There are issues using this extension together with Semantic MediaWiki - See issue 1653 on GitHub