Talk:Requests for comment/Improving extension management

Yet another concept

I find this very confusing. Extensions are essentially libraries (or PHP packages). In the olden days, those were managed by the system administrator in the form of distribution-specific packages. For some time now, composer is needed and needs to be mastered. And on top of that yet another concept? I think the way forward should be to reduce the complexity.

Wiki X requires the libraries mediawiki 1.25, mw-parserfunctions 1.3, mw-math 1.9 and mw-superextension 2.39. mw-superextension 2.39's composer.json would depend on mediawiki >= 1.23 (everything's fine) or mediawiki = 1.23 (installation/update fails). The installer spits out LocalSettings.php and an index.php that after reading LocalSettings.php calls the mediawiki library's HandleRequest() function.

Developers who are used to composer know what is happening and where to look if something does not work. --Tim Landscheidt 04:12, 26 January 2015 (UTC)

It does seem to me that Composer's "require" could handle both the supported MediaWiki version(s) and dependent extensions problems. It would however require all extensions to add Composer support. --BDavis (WMF) (talk) 08:02, 26 January 2015 (UTC)

I don't agree that extensions are essentially libraries. Libraries have code and just require an autoloader. Extensions have code with an autoloader, but they also need to register hooks, special pages, API modules, and a bunch more. Composer has never and should never be required for sysadmins who use the tarball. It is recommended for developers, but developers != sysadmins.

I'll add a section on why I don't think composer is an adequate solution to this issue. Legoktm (talk) 08:13, 26 January 2015 (UTC)

Registering can happen with a file autoload. Most Wikidata code works that way, as do for example UniversalLanguageSelector and Cldr. I'm not saying that makes composer ideal for extension management, but it can work. Adrian Heine (WMDE) (talk) 09:20, 27 January 2015 (UTC)

That works, but it's also one of the issues I outlined that composer combines the install and enable steps into one. Legoktm (talk) 17:17, 27 January 2015 (UTC)

Agreed. Adrian Heine (WMDE) (talk) 09:19, 29 January 2015 (UTC)

You are right that at the moment MediaWiki extensions differ from libraries in that they run code to essentially register themselves with MediaWiki. But that is not set in stone. If we would change the installation copy & paste instructions from:

require_once "$IP/extensions/MyExtension/MyExtension.php";

to:

require_once "$IP/extensions/MyExtension/MyExtension.php";
MyExtensionEnable();

IMHO there would be no difference at all. --Tim Landscheidt 01:10, 2 February 2015 (UTC)

Packaging

IMHO all this borders uselessness if Debian etc. packagers don't agree to use the new system. Can you email mail:mediawiki-distributors please? (I see you already sent an update for 1.25 dependencies, that's great! I hope they reply...) --Nemo 09:26, 26 January 2015 (UTC)

sent! Legoktm (talk) 18:10, 26 January 2015 (UTC)

Duplicate

Can you please merge your own Requests for comment/Configuration database 2 to this one, or summarise its outcome here? This is the fourth RfC proposing something of this kind, if I count correctly; it's hard to see or understand the progress of the discussion. --Nemo 09:26, 26 January 2015 (UTC)

Configuration database 2 is about storing all wiki settings in a database (or other backend) and allowing for on-wiki editing of those settings. This RfC just includes an enable/disable web interface, and doesn't cover any other configuration settings. Legoktm (talk) 17:41, 26 January 2015 (UTC)

So nothing at all from the discussions in the past RfC applies to this RfC? Feels like we're running in circles, just that. I'd like to discover that 4 years of discussion brought us something. :( --Nemo 17:49, 26 January 2015 (UTC)

No, I just meant that the Config DB 2 RfC was separate. I think all of the past RfCs have basically outlined the same issues (some that have now been resolved by other RfCs), but each one has had a different solution. I definitely learned things from past RfCs and incorporated them into mine though, so I don't think the 4 years of discussion were a waste. Legoktm (talk) 19:24, 26 January 2015 (UTC)

"Advanced constructs like greater than, less than, or specifying multiple constraints with a comma would not be supported"

That sounds not so useful. Extensions would either specify `*`, in which case why have versions at all, or exact versions, in which case we would end up with somthing like the fun old times for Firefox when there was no way for extensions to specify future compatibility, and every single browser update disabled all your extensions, and you had to go through them with a text editor and change version numbers...

Supporting Composer version conditions would be a much more natural option, and the code for it could probably be justed lifted from Composer. --Tgr (WMF) (talk) 08:28, 30 January 2015 (UTC)

Yup. Someone else pointed out to me that if you wanted to depend upon a feature introduced in an extension in say version 1.3.0, you'd want to do something like >1.3.0,1.*, but the currently limitations wouldn't allow for that. Legoktm (talk) 01:10, 5 February 2015 (UTC)

If we adopt the composer dependency resolver, any construct that it supports should be usable, so I've updated this part. Legoktm (talk) 02:05, 17 February 2015 (UTC)

-1 Creates technical debt

If composer is lacking in any way to handle the needs outlined here (or in the extension registration RFC), then composer should be improved.

The alternative outlined here creates a system that the MediaWiki developers are left to maintain. This will inevitably result in bitrot and reduce the reuse options available in MediaWiki. This system could use composer, true, but then it looks like you're just adding an un-necessary layer.

If we adapt Composer to our needs, we can gain the benefit of the edge cases they resolve and avoid NIH syndrome.

The points against composer and their counterpoints:

composer is CLI-only.

There is no reason composer couldn't be extended to support web access out of the box. A quick search turns up these solutions to others who have asked for non-CLI access:

• Run composer with a PHP script.
• Use composer without shell access.
• Run Composer from php script.
• NoConsoleComposer -- PHP script that can help you run composer even if you don't have SSH access to your server.

composer combines the installation process (downloading tarball and unzipping it to the right place) and the enable process (require_once/wfLoadExtension) into one step.

• File download interface
• Unzip to the right place
• re: enabling -- I don't know of any existing solutions here, but this issue looks like it might help. Jeroen knows a bit about composer and may be able to help, as well.

composer is still an alpha product, and not packaged by any major distros.

• For people installing mediawiki without shell access, this is a non-issue. They have to download mediawiki.
• This is an issue of coordinating with the system packagers. I can't find an request for packaging for Debian, for example, so someone needs to make the effort. This isn't easy work (I experienced a bit of frustration with getting MediaWiki packaging updated and it looks like it has fallen out of date again), but I don't know that this is really a Composer issue.

In the long run, I'm confident it will be to our benefit if we adopt Composer for this use case. -- ☠MarkAHershberger☢(talk)☣ 06:23, 4 February 2015 (UTC)

Thanks for the links, I added some to the RfC. How does #3603 help with the installing vs. enabling issue? I've spent the last few days looking at composer's internals and I think will be a useful starting base for this project. Legoktm (talk) 01:03, 5 February 2015 (UTC)

Thank you for being so gracious! I'm not sure, now that I look at #3603 again, how it could help. That one is as tough nut to crack. Hrm... Maybe I was thinking that the this was just something we could file a bug against Composer for and see what sort of solution they could propose. -- ☠MarkAHershberger☢(talk)☣ 03:40, 5 February 2015 (UTC)

Thinking about #3603 again: It seems like you could integrate your Extension registration work into composer some how by altering the loader. That way developers would only have to maintain one json file.

— Preceding unsigned comment added by MarkAHershberger (talk • contribs) 14:32, 13 February 2015 (UTC)

The json file for extension registration needs to be loaded on every request when the application initializes; composer.json is only needed for install/update. Seems like a bad idea to mix the two. --Tgr (WMF) (talk) 09:56, 15 February 2015 (UTC)

As a weak counter point to the idea of technical debt creation, adding a new use case to Composer just for the use of the MediaWiki community would not magically gain us leverage with the upstream project (TANSTAAFL). If anything in the short term it would make iterating on a solution more difficult as, as far as I know, there are no active MediaWiki developers with a high commit count in the Composer project. What could be a game changing thing in this direction would be a discussion involving MediaWiki and a larger group of Composer users involved in projects wanting support for a plugin/extension model added to Composer. If a common interface and use case could be found that would seem more likely to spread the maintenance burden around in the larger Composer community. --BDavis (WMF) (talk) 18:50, 16 February 2015 (UTC)

I've updated the proposal to reflect how we can leverage code from composer without turning MediaWiki extensions into composer packages. I think this addresses the issue of technical debt, since we're no longer re-inventing the wheel and completely writing our own dependency management system, we'd just need to maintain the middle layer that interacts between MediaWiki and composer, which we'd need to do anyways. Legoktm (talk) 02:02, 17 February 2015 (UTC)

Wikis on Shared Hosts

I'd like to mention that one of the great strengths of a software that runs on a traditional LAMP stack, like MediaWiki, is that it can be easily installed on a shared host.

I know it's not popular to use a shared host these days with VPS services like Amazon and Digital Ocean being increasingly popular and useful. However, the inclusion of any requirements outside of PHP and MySQL for the management of extensions increases the complexity of an installation of MediaWiki and management thereof. How many people are we leaving out who don't know about VPS, but do have an account with Bluehost or the like?

A question I might ask as we eye Composer (and similar tools) is, "Could we include new requirements in someway to reduce the technical debt for new people wanting to try out MediaWiki?" (previous unsigned comment was made by User:Ckoerner at 20:32, 4 February 2015‎)

Agreed, my proposal aims to work on all levels of hosting, including shared hosting where you might not have shell access, and IMO that should be a requirement for any extension management system. Legoktm (talk) 01:08, 5 February 2015 (UTC)

"Implementation"

Enabling extensions via special pages risks enabling a broken one and being unable to disable it.

Perhaps an mw-config/ interface would be better. Daniel Friesen (Dantman) (talk) 01:01, 17 February 2015 (UTC)

Good point. I've updated it to reflect your comment. Thanks! Legoktm (talk) 01:57, 17 February 2015 (UTC)

2015-05-13 meeting

Thanks for the summary at mailarchive:wikitech-l/2015-May/081732.html. On packaging, remember that our main page on the matter is Software bundles#Extension packages; there's also Packaging, which seems an abandoned proposal, as well as the informative Comparison of extensions in distributions. Documentation on the Debian package is at https://wiki.debian.org/MediaWiki ; per [1], it seems a new maintainer is needed. --Nemo 06:21, 14 May 2015 (UTC)

A CLI tool

Since a long time I dream about a CLI tool to install MediaWiki and extensions, and since I now propose professional support for MediaWiki I created such a tool. For now it’s only a prototype. The reasoning is behind this tool is:

• CLI-based - it can be objected that it is not user-friendly (GUI-based), but it targets people with a minimum sysadmin background
• autonomous - not to depend on external tool so that it can be simply copied on a server to, then, install and update MediaWiki; and if you don’t (resp. no more) like it, just don’t use it (resp. anymore), there are other methods to manage MW installations and installations made with this tool are very standard
• minimum requirements - Bash for now, ideally it would become Almquist Shell or Bourne Shell to be widely compatible on *nix servers
• git or tarball or composer - downloading source either from git (I personally prefer) or tarball or composer (for extensions supporting it)
• deployment-friendly - possibility to create a preproduction duplicate of a MW installation (files + database) to test it before production (details to be worked out for this part, something like git-deploy)
• no MediaWiki configuration - let MediaWiki installer do the job, or the sysadmin modify LocalSettings, or some Configuration extension; but possibly add an import-config feature (LocalSettings, images, database)
• universal - as much as possible, support installation [of files, not config] since MediaWiki 1.1 (the prototype currently does it), possibly also for extensions but it obviously encouters the same issues as discussed here (about downloading sources, dependencies)

For example you can install MediaWiki LTS + BetaFeatures in the current (empty) directory, by using their git repositories, with:

 mw-get install lts BetaFeatures --git

I intend to add a feature to activate/deactivate an extension (include_once/wfLoadExtension), improve tarball support, add installation through Composer for MW ⩾ 1.22, possibly search other downloading sources than official Wikimedia repository for extensions (parse MW.org Template:Extension or use WikiApiary), possibly also add a feature to remote install MW with SCP or FTP. I don’t intend to add a management of the config (apart activation of extensions). Any thoughts, comments?

• Repository for "mw-get"

~ Seb35 [^_^] 09:52, 21 May 2015 (UTC)

PS: why not Composer? This tool uses Composer when required (MW ⩾ 1.25 with git source) or available (MW ⩾ 1.22 with git source), but is more universal than Composer (can install MediaWiki itself, supports MW ⩾ 1.1 and intends to enable installation of extensions not registrated with Composer), it intends to support some deployment workflows, and it keeps explicit git metadata (Composer removes them explicitely, although they stay implicitely available in composer.lock). I aggree it will become less useful when Composer will become better supported by extensions and when it will become possible to MediaWiki with Composer. But for now it makes sense, and it contributes to explicitely show objective issues about extension installation/enabling (e.g. no proper way to automatically find the current LTS version, afaik). ~ Seb35 [^_^] 10:20, 21 May 2015 (UTC)