Jump to content

User:CFeng (WMF)/MachineVisionSetup

From mediawiki.org

Introduction

[edit]

This page introduces you to install mediawiki-docker and set up the MachineVision extension.

Install Docker Compose

[edit]

https://docs.docker.com/compose/install/


Clone mediawiki-docker-dev

[edit]

Clone the code to the path you would like to work on, I would recommend to clone it into ~/Documents: 

git clone git@github.com:addshore/mediawiki-docker-dev.git

And now, it is in the ~/Documents/mediawiki-docker-dev/

Please note this might be the offical one: https://www.mediawiki.org/wiki/MediaWiki-Docker

Clone MediaWiki and the skin

[edit]

Since the repository above only contains settings and scripts about setting up MediaWiki on Docker, we will need to install the MediaWiki by ourselves.

Clone it into ~/Documents: 

git clone git@github.com:wikimedia/mediawiki.git

And now, it is in the ~/Documents/mediawiki/

Also, you will need a default skin for the entire MediaWiki page to the skins folder: 

 cd ~/Documents/mediawiki/skins/</code>
 git clone git@github.com:wikimedia/mediawiki-skins-Vector.git Vector


Install Composer

[edit]

Before running the MediaWiki installation scripts, you will need the Composer. Please download and install it on your machine. 

https://getcomposer.org/download/


Run composer install for MediaWiki

[edit]

Run the followingg script to install MediaWiki 

 cd ~/Documents/mediawiki
 composer install


Create a LocalSettings.php for MediaWiki

[edit]

The LocalSettings.php contains the information that MediaWiki needs and you will need to update this file every time you add extensions.

Since the MachineVision needs a lot of extensions and other settings, here is the copy of my LocalSettings.php and please copy that into the ~/Documents/mediawiki/ 

User:CFeng_(WMF)/MachineVisionSetup/LocalSettings.php


Install extensions for mediawiki

[edit]

The LocalSettings.php has the following code: 

 wfLoadExtension( 'CirrusSearch' );
 wfLoadExtension( 'Elastica' );
 wfLoadExtension( 'MachineVision' );
 wfLoadExtension( 'UniversalLanguageSelector' );
 wfLoadExtension( 'UploadWizard' );
 wfLoadExtension( 'WikibaseCirrusSearch' );
 wfLoadExtension( 'WikibaseMediaInfo' );
 wfLoadExtension( 'WikimediaEditorTasks' );

which you will need to install the extensions manually.

Please go to the https://www.mediawiki.org/wiki/Special:ExtensionDistributor to find the extensions and copy them to the ~/Documents/mediawiki/extensions/

or, you can run the following scripts: 

 cd ~/Documents/mediawiki/extensions
 git clone git@github.com:wikimedia/mediawiki-extensions-CirrusSearch.git CirrusSearch
 git clone git@github.com:wikimedia/mediawiki-extensions-Elastica.git Elastica
 git clone git@github.com:wikimedia/mediawiki-extensions-MachineVision.git MachineVision
 git clone git@github.com:wikimedia/mediawiki-extensions-UniversalLanguageSelector.git UniversalLanguageSelector
 git clone git@github.com:wikimedia/mediawiki-extensions-Wikibase.git Wikibase
 git clone git@github.com:wikimedia/mediawiki-extensions-WikibaseCirrusSearch.git WikibaseCirrusSearch
 git clone git@github.com:wikimedia/mediawiki-extensions-WikibaseMediaInfo.git WikibaseMediaInfo
 git clone git@github.com:wikimedia/mediawiki-extensions-WikimediaEditorTasks.git WikimediaEditorTasks

Setting up Wikibase

[edit]

Include the following into composer.local.json at the ~/Documents/mediawiki/: 

{
  "extra": {
    "merge-plugin": {
      "include": [
        "extensions/Wikibase/composer.json"
      ]
    }
  }
}

and run the script to apply the changes at the ~/Documents/mediawiki/ installation 

composer install


MediaWiki-Docker environment configuration

[edit]

Copy the default.env in ~/Documents/mediawiki-docker-dev and into a new file called local.env.

Please make sure to update the mediawiki path of your installation. 

DOCKER_MW_PATH=/Users/[USERNAME]/Documents/mediawiki

Some other settings in the local.env you might need to update, for example, the PHP version you've installed, or the port has already taken by other services.


Create and start the Docker container

[edit]
cd ~/Documents/mediawiki-docker-dev
./create

The script waits up to 120 secs (4 x 30 seconds) for the database containers to initialize and respond.


Host name resolution configuration

[edit]

Adding the following configuration to your /etc/hosts 

127.0.0.1 default.web.mw.localhost # mediawiki-docker-dev
127.0.0.1 proxy.mw.localhost # mediawiki-docker-dev
127.0.0.1 phpmyadmin.mw.localhost # mediawiki-docker-dev
127.0.0.1 graphite.mw.localhost # mediawiki-docker-dev

Please note that you might need sudo when editing the hosts.

For more details of mediawiki-docker configuration, please visit: https://github.com/addshore/mediawiki-docker-dev/blob/master/README.md


MachineVision extension configuration

[edit]

Run composer install in the MachineVision extension folder: 

cd ~/Documents/mediawiki/extensions/MachineVision
composer install

and then go to the mediawiki-docker to ssh into the webserver container 

cd ~/Documents/mediawiki-docker-dev
./bash
php maintenance/update.php


Import the MachineVision mapping data

[edit]

Download the mapping data from Google: https://developers.google.com/freebase/#freebase-wikidata-mappings

Extract the downloaded file from its archive and place it somewhere that MediaWiki can access it. To import, run the following maintenance script: 

php maintenance/populateFreebaseMapping.php --mappingFile [FILE_PATH].nt

This process may take some time to complete.


Apply and enable the Google Cloud Vision API

[edit]

Go to https://console.cloud.google.com and create a project, for example, MachineVision and then click on APIs & Services on the left menu to enable the API we need.

In the API list, search Cloud vision and enable it.

To use the API, we will NEED to add credentials to the project. Once you have created a credential, you can choose the format of the credential to be JSON, and download it to your machine.

Not sure if this is associate with the service, but you might need to activate the free trial status with $300.00 credit on your account in order to activate the service successfully.

Since we have already set the credential path in LocalSettings.php, we will need to copy the credential to ~/Documents/mediawiki and rename the file as machine-vision-credentials.json to match the following setting: 

 $wgMachineVisionGoogleCredentialsFileLocation = '/var/www/mediawiki/machine-vision-credentials.json';

For more details please vision here: https://github.com/wikimedia/mediawiki-extensions-MachineVision/blob/master/README.md

Upload images and test the API

[edit]

Go to http://default.web.mw.localhost:8080/, create an account and log in. Once you have logged in, you can upload images to the MediaWiki.

After the images being uploaded, you will need to run a script to communicate with Cloud Vision API. 

cd ~/Documents/mediawiki-docker-dev
./bash
php maintenance/runJobs.php

and, check the local ImageLabel API to see if you have done all the configurations right. 

http://default.web.mw.localhost:8080/mediawiki/api.php?format=json&formatversion=2&errorformat=plaintext&action=query&generator=unreviewedimagelabels&guillimit=10&prop=imagelabels%7Cimageinfo&iiprop=timestamp%7Cuser%7Curl%7Cmime&iiurlwidth=320&uselang=en

If you see an empty response, please make sure you have set up the Cloud Vision API credential correctly.

Retrieved from "https://www.mediawiki.org/w/index.php?title=User:CFeng_(WMF)/MachineVisionSetup&oldid=3918940"