Wikiscan
 Release status: beta |
|
|---|---|
| Implementation | Database |
| Description | Edit statistics for a wiki |
| Author(s) | (Akerontalk) |
| Latest version | 0.9.2 (3-4-2021) |
| MediaWiki | 1.34+ |
| PHP | 7.2 |
| License | GPL v2 |
| Download | GitHub: Note: |
| Example | Multi site: https://wikiscan.org/, single site: https://stats.wikidex.net/ |
Wikiscan is a web-based tool that computes and displays historical statistics about a wiki site, originally for all Wikimedia Foundation (WMF) hosted wiki sites. Its About page states: "Wikiscan is not linked to the Wikimedia Foundation, the statistics published here are not official."[1]
The Wikiscan application is standalone and written in PHP. It was created in 2011 on the initiative of a voluntary user for WMF wikis, and is located at https://wikiscan.org. Most of the software development was done on a voluntary basis. Although independent of the Wikimedia Foundation, Wikiscan was part subsidized by Wikimedia Foundation and the Wikimedia France chapter association, the latter of which is currently providing server hosting for Wikiscan.
The software code was originally available on the Wikiscan.org website for download. However, it is no longer available for download. In 2021, User:Ciencia Al Poder became interested in this tool for another wiki project, and was able to download a copy of the source code before it was removed; he documented some parts of it, and did a general review to fix some issues so it is able to work on independent wiki sites.
The Wikiscan tool generates statistics by querying public data from the wiki site's database, and stored on its own database. Data displayed on the tool is not updated in real time. The statistics data must be generated on a regular basis, using cron or similar.
Download
[edit]Wikiscan source code can be downloaded from GitHub, at: https://github.com/ciencia/wikiscan/
Requisites
[edit]This tool currently needs to be installed on it's own domain or subdomain. It does not support installing on a subdirectory other than the web root.
- Webserver
- PHP 5.5 or newer, tested on PHP 7.3
- PHP modules : mbstring, curl, gd, mysqlnd, memcached
- MySQL
- It uses its own database and also needs access to the wiki's database
- Memcached
- Used for caching results
Installation
[edit]- Extract Wikiscan to a directory (example /var/www/wikiscan/)
- Create a new Mysql database (example wikiscan_stats)
- Import ctrl/stats.sql to create the tables.
- Create or configure a Mysql user with SELECT,INSERT,UPDATE,DELETE rights to the wikiscan database (example: wikiscan_user)
- Create or configure a Mysql user with SELECT rights to the wiki's database (it can be the same previously created user)
- Go to config/ directory
- Copy db_conf.sample.php to db_conf.php
- Edit db_conf.php
- Configure "db" with your Mediawiki database
- Configure "dbs" with the new stats database
- Copy local_conf.sample.php to local_conf.php
- Edit local_conf.php to configure local paths and wiki urls
- Make the directories wpstats/ cache/ and img/* writable
The site should work at this stage, but without any statistics yet.
Setting up rewrite rules
[edit]This tool generates some links that expect an existing set of rewrite rules for them to work.
apache
[edit]You can see code in the .htaccess file.
nginx
[edit]With alternate paths in Spanish:
map $request_uri $wikiscan_qs {
default "";
"~^/\?" $args;
"~^/(?:hours|horas)/?$" "menu=live&list=pages";
"~^/(?:hours|horas)/(\d+)/?$" "menu=live&date=$1&list=pages&sort=weight";
"~^/(?:hours|horas)/(\d+)/([^&/]+)/?$" "menu=live&date=$1&list=$2&sort=weight";
"~^/(?:grid|matriz)/?$" "menu=grid";
"~^/(?:grid|matriz)/(\d+)/?$" "menu=grid&date=$1";
"~^/(?:calendar|calendario)$" "menu=dates&list=pages";
"~^/(?:date|fecha)/(\d+)/?$" "menu=dates&date=$1&list=pages&sort=weight";
"~^/(?:date|fecha)/(\d+)/([^&/]+)/?$" "menu=dates&date=$1&list=$2&sort=weight";
"~^/(?:users|usuarios)$" "menu=userstats";
"~^/(?:users|usuarios)\?graphs_details=1$" "menu=userstats&graphs_details=1";
"~^/(?:user|usuario)/?$" "menu=userstats";
"~^/(?:user|usuario)/([^&]+)$" "menu=userstats&user=$1&usort=date";
"~^/(?:user|usuario)/.+$" "menu=userstats&user=_";
"~^/(?:userlist|listausuarios)/([^&]+)$" "menu=userstats&userlist=$1";
"~^/ip/?$" "menu=userstats_ip";
"~^/ip/([^&]+)$" "menu=userstats_ip&user=$1";
"~^/ip/.+$" "menu=userstats_ip&user=_";
"~^/pages_vues/?$" "menu=pageview";
"~^/pages_vues/(\d+)/(.*)$" "menu=pageview&year=$1&ns=$2";
"~^/pages_vues/(\d+)$" "menu=pageview&year=$1";
"~^/pages_vues/.+$" "menu=pageview&year=_";
"~^/ranges/?$" "menu=ranges";
"~^/range/([^&]+)$" "menu=ranges&range=$1";
"~^/range/.+$" "menu=ranges&range=_";
"~^/Derniers_utilisateurs_actifs" "menu=active_users";
}
server {
listen 80;
#listen 443 ssl;
server_name stats.mywikisite.com;
root /home/www/wikiscan;
disable_symlinks on;
access_log /var/log/nginx/wikiscan-access.log;
error_log /var/log/nginx/wikiscan-error.log;
location = / {
try_files $uri @wikiscan;
}
location / {
if ($wikiscan_qs = "") {
return 404;
}
try_files $uri @wikiscan;
}
location /img {
try_files $uri =404;
}
location /libs {
try_files $uri =404;
}
location /wikiscan {
try_files $uri =404;
}
location = /gimg.php {
fastcgi_pass unix:/run/php/php-fpm-wikiscan.sock;
include /etc/nginx/fastcgi_params;
fastcgi_param HTTPS "on";
fastcgi_param SCRIPT_FILENAME $document_root/gimg.php;
fastcgi_param QUERY_STRING $query_string;
}
location @wikiscan {
fastcgi_pass unix:/run/php/php-fpm-wikiscan.sock;
include /etc/nginx/fastcgi_params;
fastcgi_param HTTPS "on";
fastcgi_param SCRIPT_FILENAME $document_root/index.php;
fastcgi_param QUERY_STRING $wikiscan_qs;
}
}
Configuration parameters
[edit]Edit the conf/local_conf.php file, adding or updating keys of the $local_conf variable:
| Key | Description |
|---|---|
| root_path | Local filesystem path of this wikiscan installation. Example: /home/www/wikiscan |
| logo | URL of a small Wiki logo (80px tall recommended) |
| icon | URL to a favicon image |
| wikilink_icon | URL for a small icon to use as links to the wiki (defaults to WMF logo) |
| google_analytics | Google analytics account (leave empty if you don't want to track users) |
| memcache_host | Memcached host. Example: 127.0.0.1 |
| memcache_port | Memcached port. Example: 11211 |
| site_language | Language of the wiki. See Supported languages. This may determine some paths of the wikiscan site. See urlpath-menu-* language messages. Example: en. |
| interface_language | Wikiscan's interface language. See Supported languages. Example: en. |
| menus_enabled | List of menus enabled. Leave it to false to not restrict any. Implemented as security measure since there are some menus not fully developed and not reviewed. Example: ['home','live','grid','dates','userstats','userstats_ip']
|
| mw_api | API url, used to retrieve the list of namespaces, or members of a category to get users from. Example: https://en.wikipedia.org/w/api.php |
| link_page | URL to article path (page titles will be appended to this URL). Example: https://en.wikipedia.org/wiki/ |
| wiki | Local wiki configuration (when not multi site). Array with keys:
|
| groups | Local/custom group names/abbreviations. Use it if you have defined custom groups or you've localized existing group names.
Example: 'groups' => [
'rollback'=>[
'name'=>'Reversor',
'abbr'=>'R',
],
],
|
| view_about_link | Display about link. Defaults to true |
| userlist_enabled | Disable search for userlist. Defaults to false. It allows to search for user lists in the users section, by typing a page that contains a list of user links, or a category page that contains user pages. |
| robots_policy | Robots policy (for search engines). Example: noindex,nofollow |
| hits_available | Do we have hits (pageviews) information available?. Defaults to true. |
| stats_join_revision_actor_temp | use the revision_actor_temp table when building stats (need to set it to true for pre-1.35 wikis) |
| stats_join_revision_comment_temp | use the revision_comment_temp table when building stats (need to set it to true for pre-1.35 wikis) |
| log_sql | Log sql queries to the query_log table of dbg. Set it to false to not log queries, specially if you're not using a multi wiki site |
| live_hours | Array with the available hours in the last hours (live) section. Example: [24, 48]. |
Multi wiki site
[edit]The Wikiscan tool was created originally for use on a single wiki site; it was subsequently expanded to serve all WMF wiki sites under the wikiscan.org domain. In the multi wiki setup option, each wiki site has its own Wikiscan database, along with an additional database containing data to coordinate all wiki sites. Under this multi wiki setup, this tool makes heavy use of undocumented and WMF-specific database views, and also it has connections with Wikidata.
This multi wiki behavior is controlled by the 'multi' configuration parameter, that defaults to false.
Supported languages
[edit]Currently supported interface languages:
- English (en)
- French / Français (fr)
- Spanish / Español (es)
To add more languages, copy or add a file in include/languages/ with the desired ISO language code, translating from an existing one.
Known issues
[edit]- It doesn't support wiki sites with a non-empty
$wgDBprefixset (Bug #1).
Populating statistics for the first time
[edit]To update all daily stats and user stats, execute :
php ctrl/run.php fullupdate php ctrl/run.php fullupdate_months
To update last hours, execute :
php ctrl/run.php update_live
To update grid, execute :
php ctrl/run.php update_sum
Update statistics periodically
[edit]To update statistics regularly you can set up cron jobs to run several scripts:
- ctrl/run.php update_live: Updates section hours and grid. Run it several times a day. It's fast.
- ctrl/run.php update_sum: Updates calendar and users section. Run it at least once a day.
- ctrl/run.php update_misc: Updates userstats graphs. Run it at least once a week.