Extension talk:Simple Farm/Flow

Previous page history was archived for backup purposes at Extension talk:Simple Farm/LQT Archive 1 on 2015-07-10.

Hello,

I'm trying to install Simple Farm, I downloaded the extension I added the following code:

require_once "$IP/extensions/SimpleFarm/SimpleFarm.php"; require_once "$IP/FarmMembers.php";

SimpleFarm::init(); 

 /* ... Member Settings ... */

at the bottom of my LocalSettings.php.

Then, I created the FarmMembers.php file and added  :

 <?php
 # Farm members:
 $egSimpleFarmMembers = array(
    array(
              'name' => 'my_wiki_name',
              'db' => 'my_db_name',
              'addresses' => array( 'http://my_wiki_name' ),
              'maintain' => false, 
              ),
);	

Now, when I launch my wiki, i have "No wiki farm member found here!". Did I miss something? What's after /* ... Member Settings ... */? I thank you for your help.

I'm working on an extension for WikiHub that will allow users to create new wikis from a special page for their convenience.

I'm working on a script here that can handle the new wiki creation.

I've also outlined the features of the extension on my development website and plan to include a method of tracking what wikis have been created and wiki creation by user. I don't think this will be implemented as a standard MediaWiki log, but as a database query from a special page that will gather all wikis created by a given user or by all users.

To simplify tracking of wiki creation, I will make the special page only accessible from a central wiki on the farm that will be designated as the main wiki where new wikis can be created from. For some farms this could be the community or meta wiki.

This post was posted by Lieutenant S. Reznov, but signed as Inquisitor Ehrenstein/Александры реактион.

I'm wondering how I create new wikis, and I also want to use only one database.

This post was posted by Lieutenant S. Reznov, but signed as Inquisitor Ehrenstein.

Using only one DB is not supported currently (as described in the other topic already). When using multiple DBs, after you added your configuration for a new wiki, you have to run maintenance/update.php on that farm member.

Would it be simple to add the ability to use a single database?

I'm making a wiki hosting service and I'm working to make the wiki creation process as fast as possible. If each database on the farm used the same user, that would speed up the process, but it would still be an extra step.

This post was posted by Lieutenant S. Reznov, but signed as Inquisitor Ehrenstein.

I think the cost of implementation must be something around two to four working hours. I have not looked at the code for quite a while and I think there was some conceptual problem with why adding this right away was not possible, I think it was because currently the code is taking the data base name as ID for identifying the farm members. If multiple farm members should be allowed to share one database, we had to introduce some other field as the ID to identify farm members.

Around next month, I would have time implementing this if you are interested in hiring me. Of course you could also try implementing this on your own or creating a hack for this. If you submit a patch to gerrit, I would happy to review it.

I'm using multiple databases with a wildcard on the database user for the database prefix. That way the wiki creation process is simplified.

I can also add instructions to the page for how to do this and how it can be beneficial for services that need to be able to create wikis quickly.

I haven't been able to run the update script for a single wiki. What exactly is the command for it? I'm guessing from the instructions that it's

php www/extensions/SimpleFarm/maintenance/maintainFarm.php "php /home/path/to/www/maintenance/update.php" --farmonly=wikihost_mywikiBattlefield

This post was posted by Lieutenant S. Reznov, but signed as Inquisitor Ehrenstein.

Just navigate to your extensions/SimpleFarm/maintenance directory and then execute php maintainFarm.php "php ../../../maintenance/update.php --farmonly=wikihost_mywikiBattlefield.

Or even better, if only doing this for one wiki, you can simply run this from the root of your MW installation:

WIKI=wikihost_mywikiBattlefield php maintenance/update.php

Thanks. It keeps giving an exception when I try either of those options though.

This post was posted by Lieutenant S. Reznov, but signed as Inquisitor Ehrenstein.

What exception? Permission denied?

That wasn't it. There was a bunch of stuff. Isn't it not safe to post exceptions? I can take a look again and see generally what it was about.

This post was posted by Lieutenant S. Reznov, but signed as Inquisitor Ehrenstein.

It depends on the exception of course. If there is any sensible data in the exception text (like paths you don't want to have revealed, user names, passwords) then you should replace those. But in general there shouldn't be a problem with posting exceptions.

I ran

WIKI=wikihost_mywikiBattlefield php maintenance/update.php

from the root of the MediaWiki installation. I got this:

Turning off Content Handler DB fields for this part of upgrade.
...ipblocks table does not exist, skipping new field patch.
...ipblocks table does not exist, skipping new field patch.
...already have interwiki table
Missing rc_timestamp field of recentchanges table. Should not happen.
Backtrace:
#0 [internal function]: MysqlUpdater->doIndexUpdate()
#1 /home/sasha/web_servers/mywiki.ssu.lt/www/includes/installer/DatabaseUpdater.php(435): call_user_func_array(Array, Array)
#2 /home/sasha/web_servers/mywiki.ssu.lt/www/includes/installer/DatabaseUpdater.php(387): DatabaseUpdater->runUpdates(Array, false)
#3 /home/sasha/web_servers/mywiki.ssu.lt/www/maintenance/update.php(150): DatabaseUpdater->doUpdates(Array)
#4 /home/sasha/web_servers/mywiki.ssu.lt/www/maintenance/doMaintenance.php(110): UpdateMediaWiki->execute()
#5 /home/sasha/web_servers/mywiki.ssu.lt/www/maintenance/update.php(192): require_once('/home/sasha/web...')
#6 {main}

This post was posted by Lieutenant S. Reznov, but signed as Inquisitor Ehrenstein.

If you are setting up your wiki initially, not just doing some update, simply running update.php won't work I guess.

I don't quite remember how I used to set up new wikies. I guess you can either try WIKI=wikihost_mywikiBattlefield php maintenance/install.php or try disabling simple farm and then doing the usual web based setup of the wiki, afterwards re-instating the localsettings with the farm again and then adding the new member to the farm's setup.

Let me know if you have trouble or made it. I really should figure this out again and document it I guess.

Thanks. I'll try that script instead. Having to run through the install setup interface would be a bit too much though, if that's required, since I'm using the extension to run a wiki hosting service; wiki creation should be as simple as possible. Adding a database, copying and pasting a few wiki config settings, and running the install script would be enough. I can also add whatever I do to the documentation. I think there's a few other things I need to add like removing the wiki name and database name settings from LocalSettings.php.

This post was posted by Lieutenant S. Reznov, but signed as Inquisitor Ehrenstein.

I just tried running that script with the command

WIKI=wikihost_mywikiBattlefield php maintenance/install.php --pass=*************** Battlefield_Wiki Inquisitor_Ehrenstein

And I get this error:

PHP Fatal error:  Call to a member function getDB() on a non-object in /home/sasha/web_servers/mywiki.ssu.lt/www/extensions/SimpleFarm/includes/SimpleFarm.php on line 164
Fatal error: Call to a member function getDB() on a non-object in /home/sasha/web_servers/mywiki.ssu.lt/www/extensions/SimpleFarm/includes/SimpleFarm.php on line 164

This post was posted by Lieutenant S. Reznov, but signed as Inquisitor Ehrenstein.

I think I committed a patch for this error about two weeks ago to the git repo. But even then, this will still not work. I tried this myself now and you have to proceed as follows

• temporarily rename your localsettings.php
• run the install.php script, e.g. php maintenance/install.php wikiname wikiadmin --pass=adminpw --dbname=test --dbuser=user --dbpass=***. Any other parameters used here should be compliant to your actual localsettings or to your new farm member's settings (see next point)
• enter your new Wiki to your $egSimpleFarmMembers, make sure you use settings compliant to any additional parameters used in the install script
• delete the newly created localsettings.php and reinstate your old one

Now, this should really work, though, very inconvenient. I might work on a script simplifying this, though this script should be heavily tested, so I won't get around getting this done before next month.

Thanks. I'll try that. When you get a better method implemented, that will be good. I'm working on building a wiki hosting service, so I'm looking for a way of quickly implementing new wikis upon request.

This post was posted by Lieutenant S. Reznov, but signed as Inquisitor Ehrenstein.

The database now seems to be created, but I get a message saying that there is no farm member at the wiki location.

http://battlefield.mywiki.ssu.lt/Main_Page

I have this listed in the farm members file:

array(
                'name' => 'Battlefield_Wiki',
                'db' => 'wikihost_mywikiBattlefield',
                'addresses' => array( 'battlefield.mywiki.ssu.lt' ),
                'maintain' => false,
        ),

I have the database user set to use all the databases with the same prefix, so it works for all of them. Do I still need to set the user in this file?

EDIT

I'm getting a database error, even though I've added this:

array(
                'name' => 'Battlefield_Wiki',
                'wgDBprefix' => '',
                'db' => 'wikihost_mywikiBattlefield',
                'addresses' => array( 'battlefield.mywiki.ssu.lt' ),
                'maintain' => false,
        ),

EDIT

For some reason, it won't let me set database prefix per wiki.

This post was posted by Lieutenant S. Reznov, but signed as Inquisitor Ehrenstein.

Thank you, it seems that I've finally figured it out. I needed to make sure that I was setting the prefix in the installation script. I've also saved the command used for installation to reuse to make the process easier. I could probably also work on a shell script that could be used to simplify the process. Ideally I could make a script that only required inputing the database name and the wiki name.

This post was posted by Lieutenant S. Reznov, but signed as Inquisitor Ehrenstein.

That would be great!

I don't get how you did the thing with the DB prefixes though. As far as I remember, my extension does not support shared databases with several wikis but different table prefixes for technical reasons ఠ_ఠ

I'm using separate databases, but I have mw_ as the table prefix. I forgot to add this when running the install script, which resulted in having different prefixes on each database. The extension seems to only be able to handle one prefix; it can't be set per wiki.

I could put together a shell script fairly quickly, though my ultimate goal would be to make a secondary extension that adds a form for allowing users to create new wikis on their own. There would be issues with getting the extension to securely run a shell script with input from the form. I don't actually know if it's possible for PHP to run a shell script, plus I have obvious security concerns. I'll add a link on the extension page if I can get it set up.

Do you know if this extension can work with a shared user table for global accounts? It would make it much easier to manage wiki administrators, especially with a global staff group.

This post was posted by Lieutenant S. Reznov, but signed as Inquisitor Ehrenstein.

I just created a basic script for reducing wiki creation to a single command. It would work better for users to substitute variables that would remain constant during wiki creation to allow for an easier creation.

https://gist.github.com/Inquisitor-Sasha/7348768

I'll also write about putting the script into user/bin so all users need to do is navigate to the wiki directory and type make-wiki followed by the command arguments.

Eventually I'll work on implementing the script in an extension to take input from a form so that web users can create new wikis directly on their own. I'll also need to look into a way to then add the new wiki code to FarmMembers.php.

This post was posted by Lieutenant S. Reznov, but signed as Inquisitor Ehrenstein/Александры реактион.

I've created a discussion here about planning the extension to allow wiki creation from the wiki farm on the web.

https://www.mediawiki.org/wiki/Thread:Extension_talk:Simple_Farm/Developing_a_companion_extension_to_allow_wiki_users_to_create_new_wikis

This post was posted by Lieutenant S. Reznov, but signed as Inquisitor Ehrenstein/Александры реактион.

When I run the command

php maintainFarm.php "/home/sasha/web_servers/mywiki.ssu.lt/www/maintenance/update.php" --farmonly=wikihost_mywikiBattlefield

I get this

~~
~~ Simple Farm NOTE: No farm member selected in 'WIKI' environment var.
~~                   Auto-selected main member 'wikihost_mywikiBattlefield'.
~~

~~ The following 1 out of 2 farm members are selected:
~~ (1) Name: 'Battlefield_Wiki' DB: 'wikihost_mywikiBattlefield'

~~ Start maintaining selected 'Simple Farm' members:
~~ Command: '/home/sasha/web_servers/mywiki.ssu.lt/www/maintenance/update.php'

~~ (1/1) Running command for member 'Battlefield_Wiki' (WIKI=wikihost_mywikiBattlefield)...

sh: 1: /home/sasha/web_servers/mywiki.ssu.lt/www/maintenance/update.php: Permission denied


~~ Finished! All 1 selected 'Simple Farm' members have been maintained
~~ WARNING: 1 of the command executions have returned a value implying an error. See output above!

There are no tables added to the database.

This post was posted by Lieutenant S. Reznov, but signed as Inquisitor Ehrenstein.

This is the wrong command, you need to use php in front of the update script!

The extension page lists this as the content for .htaccess

# (c) 2011-2012 by Stephan Jauernick (http://stejau.org)
# Distributed under ISC license
RewriteEngine on
 
# crazy magic:
# this piece checks if we have mess with wiki stuff
RewriteCond %{REQUEST_FILENAME} wiki$
# next rule sets a env var to a value which can be used to check if files are in shared code base
RewriteRule ^/?wiki/(.*)$ - [E=REQUEST_FILENAME_W:%{DOCUMENT_ROOT}/w/$1]
# check if file is existent in shared code base
RewriteCond %{ENV:REQUEST_FILENAME_W} -d [OR]
RewriteCond %{ENV:REQUEST_FILENAME_W} -f
# if so: rewrite
RewriteRule ^/?wiki/(.*)$ /w/$1 [PT,L,QSA]
 
# standard rewrite magic:
RewriteRule ^/?wiki/[Ii]ndex\.php/(.*)$ /w/index.php?title=$1 [PT,L,QSA,B]
RewriteRule ^/?wiki/(.*)$ /w/index.php?title=$1 [PT,L,QSA,B]
RewriteRule ^/?wiki$ /w/index.php [PT,L,QSA,B]
RewriteRule ^/?images/(.*)$ /w/images/$1 [PT,L,QSA,B]
RewriteRule ^wiki/images/(.*)$ /w/images/$1 [PT,L,QSA,B]
RewriteCond %{REQUEST_URI} ^/$
RewriteRule ^/?$ /w/index.php

And I have the domain.tld/pagename setup. (yourwiki.mywiki.ssu.lt)

I changed it to this but it doesn't work.

# (c) 2011-2012 by Stephan Jauernick (http://stejau.org)
# Distributed under ISC license
RewriteEngine on

# crazy magic:
# this piece checks if we have mess with wiki stuff
RewriteCond %{REQUEST_FILENAME} wiki$
# next rule sets a env var to a value which can be used to check if files are in shared code base
RewriteRule ^/?(.*)$ - [E=REQUEST_FILENAME_W:%{DOCUMENT_ROOT}/$1]
# check if file is existent in shared code base
RewriteCond %{ENV:REQUEST_FILENAME_W} -d [OR]
RewriteCond %{ENV:REQUEST_FILENAME_W} -f
# if so: rewrite
RewriteRule ^/?(.*)$ /$1 [PT,L,QSA]

# standard rewrite magic:
RewriteRule ^/?[Ii]ndex\.php/(.*)$ /index.php?title=$1 [PT,L,QSA,B]
RewriteRule ^/?(.*)$ /index.php?title=$1 [PT,L,QSA,B]
RewriteRule ^/?$ /index.php [PT,L,QSA,B]
RewriteRule ^/?images/(.*)$ /images/$1 [PT,L,QSA,B]
RewriteRule ^wiki/images/(.*)$ /images/$1 [PT,L,QSA,B]
RewriteCond %{REQUEST_URI} ^/$
RewriteRule ^/?$ /index.php

Do you know what I should change it to?

This post was posted by Lieutenant S. Reznov, but signed as Inquisitor Ehrenstein.

This looks like you don't want the /wiki/ or /w/ part in your urls, is that right? I don't quite remember why it is or where I read it, but I remember that it is being discouraged from doing so in general, I tried it a few years ago and remember switching back to the old scheme because it wouldn't work well.

Hi, I think this is a great extension. Just to make sure that I understood correctly: This extension allows to install more than one wiki using different databases on a single server, using the same source code?! Cheers

That is right. It's not the only extension like that, there are at least two more, see Category:Wiki farm. Extension:WikiFarm though seems to have documentation in Russian only and Extension:Farmer requires changes on the MW code, also it has been reported broken quite often. Just read the documentation of Simple Farm and then you still need to use some rewrite rules on your web server, so all web-requests to the particular wikis are getting re-directed to the one MediaWiki installation with Simple Farm inside.

Cool. Heaps of cudos for you. For the very reasons you mentioned I refrained from using the two alternatives. Since I am not a coder I did not dare to try implementing the drupal-style approach on my own either. This is why I am happy to see this extension come to life. Cheers

What if you are using one database? I see the DB setting for the wiki configurations. Is it actually for a different database or is it just a table prefix?

This post was posted by Lieutenant S. Reznov, but signed as Inquisitor Ehrenstein.

Currently the extension does not support $wgDBprefix simply because I did not need it (I prefer one DB per wiki). Would not be too hard to implement this I guess.

I'm wondering how to use short url with simple farm. I'm using scriptpath-based setup. I've added this rewrite rule and it worked fine, but with no short url.

RewriteRule ^(?!wiki(?:/|$))[^/]+(?:/(.*))?$ wiki/$1

I've tried to add rewrite rules for short url but with no success.

Some more information would be helpful for analyzing your problem. It works for me but I guess there are a few other factors such as global variables that could effect a positive outcome.

Here is my configuration: a folder named "fruits", and inside it another folder named "wiki" which contains the MediaWiki files.

I have the simple farm extension installed, with FarmMembers.php containing this code:

<?php
$egSimpleFarmMembers = array(
	array(
		'name' => 'Apple',
		'db' => 'apple',
		'addresses' => 'localhost',
		'scriptpath' => '/fruits/apple',
		'maintain' => false,
	)
);
?>

apache2.conf file has the following configuration:

<Directory /var/www/html/fruits>
  Options +FollowSymlinks
  <IfModule mod_rewrite.c>
    RewriteEngine On
    RewriteRule ^(?!wiki(?:/|$))[^/]+(?:/(.*))?$ wiki/$1
  </IfModule>
</Directory>

This all works correctly but problems arise when I want to add short URL to it using the apache2.conf file. I tried to add short URL rewrite rules after the simple farm one, and tried to combine both, but with no success. Could you please tell me which rewrite rules did you write to get it to work successfully?

To make things clear, what I want to accomplish is a URL like this: localhost/fruits/apple/Main_Page and not localhost/fruits/apple/index.php/Main_Page .. i.e. removing the index.php part

Please read the comment provided together with the rewrite rule at Simple Farm - Setting-up Rewrite Rules for scriptpath based farm members.

It states No '/' allowed within the wiki path when using the rule, referring to RewriteRule ^(?!wiki(?:/|$))[^/]+(?:/(.*))?$ wiki/$1.

So if you are aiming for /fruits/apple and /fruits/strawberry instead of /apple and /strawberry you need to modify that rule accordingly, e.g. as in the example given in the very same code example from the section above: RewriteRule ^(?:fruits|fruits/apple|fruits/strawberry)(?:/(.*))?$ wiki/$1

Not sure you have anything else not configured accordingly, just try to change that first.

Hi Danwe, I am pretty much done with my wiki farm. I had to alter your setup, since I use one shared database (per domain) for all wikis, and differentiate between them with prefixes. Since the prefix is not part of your extension, I use virtual database names instead. This works since you do not access the database within your extenson, so I can re-define the real database name after the farm is intialized.

The setup works fine but I have one issue left about not displaying any images or logos. I outlined a sample of my configuration on a separate page in oder to avoid messing up your talk page. I assume the issue is based on the rewrites in my .htaccess, but I can't figure it out. Do you mind looking at my my configuration and giving me a hint where I failed? Anybody else who is reading this, how can I modify my rewrites to access the images?

Hi Danwe, I found it eventually. The [B] opton in the rewrite rules escape the necessary colon between namespace and filename. I left it out, and voila: it works now :)

I'll look into this as well, perhaps I have to improve the rewrite rules. About DB prefixes: You could probably define a custom option in your $egSimpleFarmMembers entries, for example dbprefix, and then do Manual:$wgDBprefix = SimpleFarm::getActiveMember()->getCfgOption( 'dbprefix' ); I guess it would be problematic with all members having the same db option though, especially when it comes to maintenance. I will think about improving on this, hasn't been a priority though since I always use separate DBs for my wikis.

Hi, that is really great, and I appreciate not only the work you did but also you being just in time for me merging my separate wikis to a farm. I like especially that it does not force one to a special configuration; instead you can use wikis within the same or on different databases, and the same applies to the virtual addresses as you described.

Before I start using it, I have one question left though regarding the shared directory structure. I know how to set up separate /images directories per wiki if needed, and it is nice that I do not even have to think about symlinking /extensions, but the question is about directories like /cache. In former versions of wiki farms (e.g. with symlinks) it was documented that it shall not be shared between wikis. Did that change? I did not see any code related to that topic in your solution, but may have missed it. Do you have any (bad/good) experience with sharing the cache folder?

Hello Surak! I am not 100% sure about caching but there is this "cache" directory in the mw root but it is always empty and it seems to have no functionality. Also, the variable which refers to the cache ( $wgCacheDirectory is disabled by default, and if you want to enable it, just make sure that the path is different for all of your farm members, e.g. by using $wgCacheDirectory = "/cache/{$wgDBname}". About $wgFileCacheDirectory, this depends on $wgUploadDirectory anyhow (which is set by the extension to be {$IP}/images/images_{$wgDBname} by default.

Thanks, Danwe, that makes absolutely sense.