Wikiscan

Manual on MediaWiki Tools

Wikiscan Release status: beta
Implementation	Database
Description	Edit statistics for a wiki
Author(s)	(Akerontalk)
Latest version	0.9.2 (3-4-2021)
MediaWiki	1.34+
PHP	7.2
License	GPL v2
Download	GitHub: project page git repository URL [help ] commit history Note: No localisation updates are provided by translatewiki.net .
Example	Multi site: https://wikiscan.org/, single site: https://stats.wikidex.net/

Wikiscan is a web-based tool that computes and displays historical statistics about a wiki site, originally for all Wikimedia Foundation (WMF) hosted wiki sites. Its About page states: "Wikiscan is not linked to the Wikimedia Foundation, the statistics published here are not official."[1]

The Wikiscan application is standalone and written in PHP. It was created in 2011 on the initiative of a voluntary user for WMF wikis, and is located at https://wikiscan.org. Most of the software development was done on a voluntary basis. Although independent of the Wikimedia Foundation, Wikiscan was part subsidized by Wikimedia Foundation and the Wikimedia France chapter association, the latter of which is currently providing server hosting for Wikiscan.

The software code was originally available on the Wikiscan.org website for download. However, it is no longer available for download. In 2021, User:Ciencia Al Poder became interested in this tool for another wiki project, and was able to download a copy of the source code before it was removed; he documented some parts of it, and did a general review to fix some issues so it is able to work on independent wiki sites.

The Wikiscan tool generates statistics by querying public data from the wiki site's database, and stored on its own database. Data displayed on the tool is not updated in real time. The statistics data must be generated on a regular basis, using cron or similar.

Download

Wikiscan source code can be downloaded from GitHub, at: https://github.com/ciencia/wikiscan/

Requisites

This tool currently needs to be installed on it's own domain or subdomain. It does not support installing on a subdirectory other than the web root.

• Webserver
• PHP 5.5 or newer, tested on PHP 7.3
  • PHP modules : mbstring, curl, gd, mysqlnd, memcached
• MySQL
  • It uses its own database and also needs access to the wiki's database
• Memcached
  • Used for caching results

Installation

1. Extract Wikiscan to a directory (example /var/www/wikiscan/)
2. Create a new Mysql database (example wikiscan_stats)
3. Import ctrl/stats.sql to create the tables.
4. Create or configure a Mysql user with SELECT,INSERT,UPDATE,DELETE rights to the wikiscan database (example: wikiscan_user)
5. Create or configure a Mysql user with SELECT rights to the wiki's database (it can be the same previously created user)
6. Go to config/ directory
7. Copy db_conf.sample.php to db_conf.php
8. Edit db_conf.php
   1. Configure "db" with your Mediawiki database
   2. Configure "dbs" with the new stats database
9. Copy local_conf.sample.php to local_conf.php
10. Edit local_conf.php to configure local paths and wiki urls
11. Make the directories wpstats/ cache/ and img/* writable

The site should work at this stage, but without any statistics yet.

Setting up rewrite rules

This tool generates some links that expect an existing set of rewrite rules for them to work.

apache

You can see code in the .htaccess file.

nginx

With alternate paths in Spanish:

map $request_uri $wikiscan_qs {
        default "";
        "~^/\?" $args;
        "~^/(?:hours|horas)/?$" "menu=live&list=pages";
        "~^/(?:hours|horas)/(\d+)/?$" "menu=live&date=$1&list=pages&sort=weight";
        "~^/(?:hours|horas)/(\d+)/([^&/]+)/?$" "menu=live&date=$1&list=$2&sort=weight";
        "~^/(?:grid|matriz)/?$" "menu=grid";
        "~^/(?:grid|matriz)/(\d+)/?$" "menu=grid&date=$1";
        "~^/(?:calendar|calendario)$" "menu=dates&list=pages";
        "~^/(?:date|fecha)/(\d+)/?$" "menu=dates&date=$1&list=pages&sort=weight";
        "~^/(?:date|fecha)/(\d+)/([^&/]+)/?$" "menu=dates&date=$1&list=$2&sort=weight";
        "~^/(?:users|usuarios)$" "menu=userstats";
        "~^/(?:users|usuarios)\?graphs_details=1$" "menu=userstats&graphs_details=1";
        "~^/(?:user|usuario)/?$" "menu=userstats";
        "~^/(?:user|usuario)/([^&]+)$" "menu=userstats&user=$1&usort=date";
        "~^/(?:user|usuario)/.+$" "menu=userstats&user=_";
        "~^/(?:userlist|listausuarios)/([^&]+)$" "menu=userstats&userlist=$1";
        "~^/ip/?$" "menu=userstats_ip";
        "~^/ip/([^&]+)$" "menu=userstats_ip&user=$1";
        "~^/ip/.+$" "menu=userstats_ip&user=_";
        "~^/pages_vues/?$" "menu=pageview";
        "~^/pages_vues/(\d+)/(.*)$" "menu=pageview&year=$1&ns=$2";
        "~^/pages_vues/(\d+)$" "menu=pageview&year=$1";
        "~^/pages_vues/.+$" "menu=pageview&year=_";
        "~^/ranges/?$" "menu=ranges";
        "~^/range/([^&]+)$" "menu=ranges&range=$1";
        "~^/range/.+$" "menu=ranges&range=_";
        "~^/Derniers_utilisateurs_actifs" "menu=active_users";
}

server {
        listen 80;
        #listen 443 ssl;
        server_name stats.mywikisite.com;
        root /home/www/wikiscan;

        disable_symlinks on;

        access_log /var/log/nginx/wikiscan-access.log;
        error_log /var/log/nginx/wikiscan-error.log;

        location = / {
                try_files $uri @wikiscan;
        }

        location / {
            if ($wikiscan_qs = "") {
                return 404;
            }
            try_files $uri @wikiscan;
        }

        location /img {
                try_files $uri =404;
        }
        location /libs {
                try_files $uri =404;
        }
        location /wikiscan {
                try_files $uri =404;
        }

        location = /gimg.php {
                fastcgi_pass unix:/run/php/php-fpm-wikiscan.sock;
                include /etc/nginx/fastcgi_params;
                fastcgi_param HTTPS "on";
                fastcgi_param SCRIPT_FILENAME $document_root/gimg.php;
                fastcgi_param QUERY_STRING $query_string;
        }

        location @wikiscan {
                fastcgi_pass unix:/run/php/php-fpm-wikiscan.sock;
                include /etc/nginx/fastcgi_params;
                fastcgi_param HTTPS "on";
                fastcgi_param SCRIPT_FILENAME $document_root/index.php;
                fastcgi_param QUERY_STRING $wikiscan_qs;
        }
}

Configuration parameters

Edit the conf/local_conf.php file, adding or updating keys of the $local_conf variable:

Key	Description
root_path	Local filesystem path of this wikiscan installation. Example: /home/www/wikiscan
logo	URL of a small Wiki logo (80px tall recommended)
icon	URL to a favicon image
wikilink_icon	URL for a small icon to use as links to the wiki (defaults to WMF logo)
google_analytics	Google analytics account (leave empty if you don't want to track users)
memcache_host	Memcached host. Example: 127.0.0.1
memcache_port	Memcached port. Example: 11211
site_language	Language of the wiki. See Supported languages. This may determine some paths of the wikiscan site. See urlpath-menu-* language messages. Example: en.
interface_language	Wikiscan's interface language. See Supported languages. Example: en.
menus_enabled	List of menus enabled. Leave it to false to not restrict any. Implemented as security measure since there are some menus not fully developed and not reviewed. Example: ['home','live','grid','dates','userstats','userstats_ip']
mw_api	API url, used to retrieve the list of namespaces, or members of a category to get users from. Example: https://en.wikipedia.org/w/api.php
link_page	URL to article path (page titles will be appended to this URL). Example: https://en.wikipedia.org/wiki/
wiki	Local wiki configuration (when not multi site). Array with keys: url: URL to the wiki's homepage site_host: Wiki site name
groups	Local/custom group names/abbreviations. Use it if you have defined custom groups or you've localized existing group names. Example: 'groups' => [ 'rollback'=>[ 'name'=>'Reversor', 'abbr'=>'R', ], ],
view_about_link	Display about link. Defaults to true
userlist_enabled	Disable search for userlist. Defaults to false. It allows to search for user lists in the users section, by typing a page that contains a list of user links, or a category page that contains user pages.
robots_policy	Robots policy (for search engines). Example: noindex,nofollow
hits_available	Do we have hits (pageviews) information available?. Defaults to true.
stats_join_revision_actor_temp	use the revision_actor_temp table when building stats (need to set it to true for pre-1.35 wikis)
stats_join_revision_comment_temp	use the revision_comment_temp table when building stats (need to set it to true for pre-1.35 wikis)
log_sql	Log sql queries to the query_log table of dbg. Set it to false to not log queries, specially if you're not using a multi wiki site
live_hours	Array with the available hours in the last hours (live) section. Example: [24, 48].

Multi wiki site

The Wikiscan tool was created originally for use on a single wiki site; it was subsequently expanded to serve all WMF wiki sites under the wikiscan.org domain. In the multi wiki setup option, each wiki site has its own Wikiscan database, along with an additional database containing data to coordinate all wiki sites. Under this multi wiki setup, this tool makes heavy use of undocumented and WMF-specific database views, and also it has connections with Wikidata.

This multi wiki behavior is controlled by the 'multi' configuration parameter, that defaults to false.

Supported languages

Currently supported interface languages:

• English (en)
• French / Français (fr)
• Spanish / Español (es)

To add more languages, copy or add a file in include/languages/ with the desired ISO language code, translating from an existing one.

Known issues

• It doesn't support wiki sites with a non-empty $wgDBprefix set (Bug #1).

Populating statistics for the first time

To update all daily stats and user stats, execute :

php ctrl/run.php fullupdate
php ctrl/run.php fullupdate_months

To update last hours, execute :

php ctrl/run.php update_live

To update grid, execute :

php ctrl/run.php update_sum

Update statistics periodically

To update statistics regularly you can set up cron jobs to run several scripts:

• ctrl/run.php update_live: Updates section hours and grid. Run it several times a day. It's fast.
• ctrl/run.php update_sum: Updates calendar and users section. Run it at least once a day.
• ctrl/run.php update_misc: Updates userstats graphs. Run it at least once a week.