Extension:VisualData/Maintenance scripts

From mediawiki.org

Currently VisualData offer the following maintenance scripts:


RebuildData[edit]

RebuildData can be used in the following way:

php extensions/VisualData/maintenance/RebuildData.php

it drops all the VisualData tables and completely rebuilds them based on the contents of wiki articles. As mentioned above it is particularly important that the "source of truth" are the data stored in wiki articles rather than in the tables of the extension, since the data-structure of the tables of the extensions may change at any time, and there is no guarantee that the data are preserved in the original form. It is also particularly important that the data are stored in wiki articles (as json content model) so that sysops or authorized users have direct access to them.

Within such framework, protection of sensitive data may be achieved either using preventing access extensions, or more adequately using a private wiki where to store sensitive information and a public wiki where to store public information. By this point of view future versions of the extensions may allow inter-wiki queries and/or form submissions.


ImportData[edit]

ImportData can be used in the following way:

php extensions/VisualData/maintenance/ImportData.php --file "[path-to-the-following-file]/countries+states+cities.json" --schema Country --pagename-formula "Data:countries/<name>" --main-slot

it allows to bulk-import json data to your wiki and to create an article for each of them based on the pagename-formula. For instance the previous command creates 250 wiki pages based on this data each of them with all the data related to each country, for a total of more than 620,000 imported entries.

The data are created in the main slot (with the parameter --main-slot) because no wikitext is expected to be associated to the article.

This way the wiki becomes a repository for all your data, and the specific VisualData database only a way to query the data that can be rebuilt anytime, where the "source of truth" is the standard Mediawiki database and the wiki article itself.


DeleteRegex[edit]

DeleteRegex can be used in the following way:

php extensions/VisualData/maintenance/DeleteRegex.php --regex "Data:countries/.+"

As the name and the example suggest, it allows to bulk-delete wiki articles based on a regex.


See also[edit]