Global templates/Proposed specification
This is a proposal for the functional requirements for global templates and modules.
You can also read a one-page version of this proposal.
This is not a project that is being executed or planned to be executed by anyone at any defined point in time, at least not yet. This is just an idea, albeit a very detailed one.
The eventual goal is to form a cross-team and cross-project commitment on implementing these things, with proper architecture, product and project management, community engagement, etc.
This document does not try to go into the details of technical implementation in terms of storage, caching, delivery, PHP code design, etc. It only tries to define the requirements of how this feature will work from the point of view of the users:
- People who create and maintain templates and modules.
- People who create and edit pages that transclude templates and modules. This includes all editors and all kinds of pages:
- All levels of experience: from those who are completely new to those who have made thousands of edits
- All kinds of editing tools: wiki syntax editing, Visual Editor, Content Translation, and others (even bot operators)
- All wikis: Wikipedia, Wiktionary, Wikivoyage, Wikidata, Incubator, etc., and any new future projects
- All languages: English, French, Russian, Spanish, Armenian, Persian, Zulu, Manipuri, etc.
- All kinds of pages: Wikipedia articles, articles talk pages, user talk pages, community discussion pages, WikiProject pages, categories, template documentation pages, etc. {{🌎🌍🌏}}
Elevator pitch
[edit]A large part of the functionality of Wikimedia sites is implemented in templates and Lua modules. In their current form, they cannot be shared across different wikis and languages. Because of this they are hard to integrate with modern ways of creating and editing articles, such as Visual Editor, Wikidata, and Content Translation. They are also hard to adapt to mobile devices. This causes waste of contributors’ effort, and difficulties for new editors and smaller projects. It must become possible to share them across wiki sites, similarly to Commons images. This will make software development faster and more robust, and it will also let editors focus more on writing. {{🌎🌍🌏}}
The Problem
[edit]General comment: Unless noted otherwise, all references to “templates” also apply to Scribunto Lua modules.
Templates implement features of Wikimedia sites. Some of these features are highly prominent, especially the infoboxes, the references, “citation needed”, and many others. For a more comprehensive list of functions implemented using templates, see Taxonomy. All the readers see them, and all the editors run into them in almost every edit action. In addition, they implement many of the sites’ internal community management features: requests for deletion, requests for unblocking, expressing support in discussions, sorting articles for WikiProjects, etc.
Templates provide an efficient mechanism to quickly design, deploy, and use repetitive pieces of text and markup in many pages. However, templates also have several acute usability problems for all kinds of editors.
Difficulties for all editors in all wikis
[edit]Wiki syntax editors
[edit]Templates are often difficult to understand for editors in wiki syntax. People who are experienced with using a particular template will probably identify it and be able to edit a page that includes it. However, editors who are not familiar with this template will have to look up its documentation when they encounter it, even if they are generally experienced with editing and other templates. And editors who have low experience will be baffled by the cryptic text with curly braces, pipe characters, equal signs, etc.
Using a feature that is implemented as a template requires knowing the template’s name and typing it in curly braces ({{}}) or copying it from another page. This is not obvious to new users, and veteran users have to learn each new template separately, too.
Some wikis have gadgets that add buttons that insert templates that are common in that project to the editing toolbar. These are different in each wiki, even though a lot of the templates have similar functionality across projects and languages.
Visual Editor users
[edit]VisualEditor users have some advantages with using templates, however there are many issues with them there as well. In particular, there is a similar discoverability issue: In Visual Editor, all the templates’ functionality is hidden behind the “Insert → Template” menu item, and the user has to know the template name before using it.
Visual Editor’s “Insert” menu has items for Math formulas, Egyptian Hieroglyphs, Musical scores, and some other functions that are implemented as extensions, but it doesn’t have items such as “Infobox”, “Citation needed”, “Unit conversion”, “Quotation”, etc. All the templates are the same kind of generic item.
There is one notable exception: Some wikis have the “Cite” button, which inserts footnotes with reference templates. However, it is an exception that proves the rule. It requires manual configuration for even basic functionality, this configuration is separate in each wiki, and as a consequence many wikis don’t have this button at all. Another comparable exception, which was added in late 2019, is special support for “Citation needed” templates, but this also needs some custom configuration on each and every wiki to actually work.
Difficulties for editors who write in more than one project
[edit]Many templates exist in one project, but not in others, and often a template is available, but in a different form. Because of this, it’s difficult or impossible to reuse skills that were acquired in one project: the functionality that the template provides is sometimes unavailable, and sometimes it works differently. This applies not just to wikis in different languages, but also to different wikis in the same language, for example, English Wikipedia and English Wikisource.
For people editing in different languages, templates make translation harder. When translating a page, templates are much harder to handle than the article text (“prose”), whether the translation is done manually or with Content Translation. Users often have to skip the template or to correct it after the article was published. This also causes abandoning translations in progress, because template translation looks intimidating.
The most commonly reported issues in Content Translation are about templates.
Content Translation has a template adaptation feature, which automates some parts of this process, but it works only if a corresponding template exists in both languages, and if all the parameters were meticulously mapped by the template maintainers. This must be done for each template in each language separately and manually, and continuously maintained when the source template changes. This happens even though the templates’ function across the languages is the same most of the time.
Ideally, the templates and their parameters should be transferred to the translated page almost completely automatically, so that the translators could focus on writing the prose, as writing the prose is the area where human work is most needed.
A template can be exported from one wiki to another, but once this is done, the template becomes a forked copy. It either stays in the state in which it was exported, or continues to be developed separately, causing incompatibility. Sometimes people keep maintaining the different copies, but this is not robust and cannot scale for the hundreds of wikis that we have.
Template parameters can have the same functionality, but different names in different wikis. They can be adapted using TemplateData aliases, but this is a suboptimal hack: it is not what TemplateData was originally made for, and it has to be done manually for each language pair.
Templates mix together algorithmic logic, human-readable text strings, and formatting. Because of this there is no robust way to translate the templates’ user interface strings, as it is done with MediaWiki core and extensions.
Difficulties for editors in smaller wikis
[edit]A new wiki project is created by installing core MediaWiki and enabling a default set of extensions. In practice this doesn’t create a level playing field because many key features of larger wikis are implemented in templates: infoboxes, citations, maintenance hatnotes (such as {{Unreferenced}}), etc.
Difficulties for software developers
[edit]It is difficult for developers of MediaWiki core, extensions, bots, and other tools that analyze, generate, or modify wiki page content to build features that depend on the presence of certain templates in a wiki. Developers of extensions such as GrowthExperiments, PageTriage, ContentTranslation, some components of Wikibase, and many others, have to either test them in production, which is a bad idea, or to import the templates to their local wikis or an online test wiki.
Researchers who get data about wiki content based on templates have to write their analysis code for each wiki separately, and sometimes they end up doing for only one wiki. A notable example is using English Wikipedia’s WikiProject templates to analyze page topics and assess article quality. {{🌎🌍🌏}}
Assumptions
[edit]Extensions vs Templates: Similarities and differences
[edit]One of the main assumptions of this project proposal is that templates and modules are very similar to MediaWiki core and extensions: They are software, and they implement features that the editors community needs. In particular, since templates are usually developed by the editors themselves, it’s obvious that the community really needs them. The big differences between lie in how they are developed, localized, and deployed.
Templates and modules should become similar to extensions in some key properties that they currently lack, and preserve some good properties that extensions lack. (For ease of understanding by English readers, in the table, examples of templates are given from the English Wikipedia. They could also come from any other wiki and any other language.)
| Property | Core and extensions | Templates | What should be done about this with templates? |
|---|---|---|---|
| Implement special kind of content | Yes.
image, math, hieroglyphics, score, basic ref |
Yes. | |
| └ Special kind of content is easy to insert into pages | Yes (using the toolbar). | No (unless you learned how to do it). | Should be changed. |
| └ Special kind of content is easy to adapt to mobile devices | Mostly yes. | Sometimes. | Should be changed. |
| └ Special kind of content is easy to translate as part of an article | Mostly yes. | No. | Should be changed. |
| Implement page formatting features | Yes.
gallery, poem |
Yes. | |
| Implement community workflow processes | Yes.
Abuse filter, Check user, Move |
Yes. | Should be easy to reuse, but not forced. |
| Easy to deploy initial version | No. | Yes. | Should be preserved. |
| Easy to deploy changes | No. | Yes. | Should be preserved. |
| Easy to localize | Yes. | No. | Should be changed. |
| Useful on multiple wikis | Often. | Often. | |
| Used only on a limited number of wikis | Sometimes. | Sometimes. | |
| Easy to install and use on multiple wikis | Yes. | No. | Should be changed. |
| Easy to use on a new, recently created wiki | Yes. | No. | Should be changed. |
Templates and modules development skills
[edit]Another important set of assumptions on which this proposal is based is the following:
- The skills for developing templates and modules are non-trivial. Both templates and modules have numerous obscure features.
- Even though many of the sites’ most notable features are implemented as templates and modules, these skills are often unnoticed, underappreciated, and taken for granted.
- There are dozens of people who have these skills, and they edit many wikis. They usually focus on their home wiki and relatively rarely communicate with contributors from other wikis or other languages. Even though the underlying technology is the same everywhere, there is no true global community of template developers that would be comparable to the global community of MediaWiki core and extensions developers. There are cases of cross-wiki collaboration on certain templates, but they are inconsistent.
- There are also many wikis in which there are no editors who have these skills. They either copy templates and modules from other wikis without fully understanding how they work and without the ability to localize and maintain them effectively, or they don’t use templates at all.
This situation is far from optimal. The template and module developers’ skills need more appreciation. They develop truly needed features, and they are embedded in their editors communities. In wikis in many languages, the template developers come up with innovative features for structured content, data presentation, and modularization. These innovations could be useful in many wikis, but currently there is no convenient mechanism to achieve this.
And of course, any solution for these problems must not come up with new technologies that will discard the many years of hands-on experience that the template maintainers have acquired. Hence, there must be as few changes as possible in the syntax for developing templates and modules. The things that need to change are the way in which they are deployed and propagated across wikis, and the way in which the human-readable strings in them are localized (translated). {{🌎🌍🌏}}
The proposed solution: Summary
[edit]There are already many features of MediaWiki that are global across wikis:
- images (using Commons)
- blocking
- user accounts (CentralAuth)
- preferences
- user pages
- user JS and CSS pages
- global watchlist is in active development as of late 2020 (GlobalWatchlist)
- and some others.
It must be possible to store templates and modules in a global repository, too, and to localize them as robustly as it is done with extensions.
Global templates and modules will empower template maintainers in all wikis by letting them collaborate more easily on developing the templates’ code.
Global templates and modules will empower translators and localizers by allowing them to focus on translating just the user interface strings (“messages”), without having to look for strings in the code, and by allowing them to use the same skills and tools for translating templates and MediaWiki extensions.
Global templates and modules will empower content editors in all wikis to write and translate content that uses these templates without having to dive into the differences and to relearn different rules and skills in each wiki.
The syntax for developing templates and modules, and the general template maintenance and deployment cycle will not change, so all the skills that the template maintainers have acquired over the years will remain relevant.
All wikis will be able to use the global templates, but not forced to do this. Communities will keep their capability to override any global functionality, design, workflows, and data.
Localizing templates will be as convenient as localizing MediaWiki extensions. {{🌎🌍🌏}}
The proposed solution: Detailed requirements
[edit]Templates must be able to be semantic and global
[edit]Semantic means that other software components, especially Visual Editor and Content Translation, must have a general way to understand that a template exists and that it provides certain functionality, so that it will be possible to insert it into a page as an infobox, a citation, a maintenance tag, etc., and not only as a generic template. Currently, the closest thing to making templates semantic is TemplateData, but it only describes the template’s parameters. It doesn’t, for example, help Visual Editor add an “Insert infobox” button to the toolbar.
Global means that a template’s code must be maintained in one place and usable in all wikis.
Making templates semantic
[edit]Templates have never been robustly semantic, in the sense of being easy to handle by software that processes pages.
There are only a few examples of templates that were made semantic:
- Various reference templates, which are usable from the Visual Editor toolbar “Cite” button. They require writing a lot of separate code to configure Citoid on each wiki that wants to use them.
- “Citation needed”, which was adapted to Visual Editor in late 2019 (task T211243). It also requires configuration on each wiki. For example: English, Hebrew, Slovene. As of this writing, French, Spanish, and most other languages are not configured for this, even though they have templates of this kind.
- Templates for mentioning users in the Flow extension, which require local configuration, too.
- Some dump processing and research tools can parse English Wikipedia’s WikiProject page assessment templates, which are usually added to talk pages.
- The GrowthExperiments extension suggests editors to perform certain tasks in articles based on the templates that are transcluded in them. The template names have to be configured manually by writing JSON files separately in each wiki. For example: Czech, Vietnamese, Korean, Arabic.
- The PageTriage extension is configured to work with English Wikipedia’s hatnote templates (also known as “tags”).
In the case of PageTriage, the extension essentially hard-codes a single wiki’s templates, making it unusable in other wikis without a significant rewrite. Even if the on-wiki configuration step is small, as it is with the Flow example, it still needs to be done. This doesn’t scale well for the 900 wikis that Wikimedia has, and the thousands that it will have in the future.
These things should be global by default, so that they will be immediately usable in at least a basic default configuration on all wikis at once by extensions, bots, dump analyzers, etc.
Storage and delivery
[edit]The global templates and modules can be stored in a central wiki (Meta, Commons, or a whole new wiki), and it can even be Gerrit or another repository.
The best solution is probably creating a new wiki that will store them, without getting mixed up with images, general community discussion, etc.
Using Gerrit as storage for templates and modules code is technically possible, but it would lose an important element of accessibility for template maintainers: editing a template in a wiki page is much easier and familiar to the vast majority of template maintainers than making Git commits and waiting for code review. Therefore, Gerrit probably shouldn’t become a way for storing the template and module code, at least not the primary one.
Global templates and modules must be stored in a common repository that can be edited by most wiki editors. Rules about blocking and special permissions should initially be similar to the rules in other wikis: everything should be open by default, and it must be possible to protect very common, sensitive, or frequently-vandalized templates. More detailed rules about protection levels can be developed by the editors community later.
The code of the templates in the central repository will use the generic English names of tags such as <section>, parser functions such as {{#ifexist}} or {{#invoke}}, and magic words such as {{DISPLAYTITLE}}.
How the templates are delivered to the target wikis is a question of internal engineering and architecture, as long as the other requirements are addressed. These questions were discussed in the past by some platform developers, for example around the Shadow namespaces project. This document tries to address related questions of how it works for the user who edits a page that uses a template, or who maintains the template itself—how to write it in a localizable way; how is it translated; how is it locally customized; etc. These questions weren’t addressed sufficiently in the previous architectural discussions on the topic.
Templates must remain easy to modify
[edit]An important feature of how templates currently work is that they are edited just like wiki pages and immediately become functional after publishing without review or deployment. This is somewhat dangerous because a bad edit can ruin many pages, but the fact is that it works mostly well.
This ease must be preserved. Community members who maintain templates will most likely reject moving to a new system that will require them to learn new skills and drag every change through an exhausting review and deployment phase. This probably means that storing the templates in Gerrit is not going to work, unless, perhaps, the process for review and deployment will be much easier than it is for extensions.
It must be possible to make some templates non-global
[edit]Not all templates should be forced to be global.
In fact, some templates should be local because they implement a functionality that is unique to a certain language. By their nature, such templates don’t need to be translated, and there should be a way to give a hint to both human editors and translation tools (such as Content Translation) that they don’t need to be adapted, and can be skipped or substituted. This is a part of the effort to make templates more semantic.
It must be possible to override some functionality or appearance of a global template
[edit]No community should feel that a functionality is imposed on it by some powerful external player, like the English Wikipedia community, the Wikidata community, the WMF, or anybody else. Global templates should be developed and used collaboratively for the common benefit. Most of the time, it should work for everybody.
Sometimes some communities will have strong opinions about wanting to have particular functionality or design that will be different in their language or project, or to show an infobox with information that is different from what is shown in other projects, or not to show it at all. The capability to override things locally must be allowed from the start. (Or rather, it must not be taken away.)
A global template must be immediately usable in every wiki
[edit]Just like a global user page is immediately available in every wiki in which there is no local user page, every template or module that is created on the global infrastructure must be immediately usable in every wiki.
This must not require any extra steps, such as copying wiki pages, creating wrapper templates with a local name, administrator intervention, waiting for hours for caches to refresh, etc.
After the central version is updated, the updated version will be immediately shown everywhere. To prevent vandalism, the editors community will develop policies on permissions and protection levels.
If the user interface strings (also known as “messages”) were not translated, the template will nevertheless be usable, and the strings will be shown in the fallback language. See the sections on localization for more details.
Localization of user-facing strings
[edit]It must be possible to translate all user-facing strings
[edit]The user interface strings (messages) of core MediaWiki, extensions, and some external tools such as Pageviews are translated conveniently and robustly on translatewiki.net. This localization process is familiar to at least some editors in all languages.
It’s currently not possible to do the same with templates. Multilingual sites such as Commons and mediawiki.org have the “TNT” system for translating some templates, but it’s very complicated and cannot be reused for Wikipedia, Wikisource, etc. In 2021, this became more robust thanks to the “language-aware transclusion” feature, but it still needs some improvements.
Ideally, it should be possible to translate templates just like core and extensions, using a wiki with the Translate extension.
The translated string must become usable immediately after the translation is submitted using the Translate interface.
It can be possible to edit the user interface strings in raw wiki pages, but ideally they should be edited primarily through a dedicated translation interface.
Translators should be able to focus on translating nothing but text. Seeing any code around it makes it difficult for people who are not experienced with programming and JSON files to contribute easily. In addition, editing translations into languages that are written from right to left in raw text files is extremely inconvenient. The Translate extension already addresses all of these issues.
The template documentation pages must be translatable as well. It’s mostly enough to do it using the page translation feature of the Translate extension, but it may require some adaptations.
The language in which the strings are shown to the user
[edit]Templates are primarily used when they are integrated into content, so by default the translated messages must be shown in the wiki’s content language.
Some templates, however, are used as UI elements. Therefore, perhaps it makes sense to also allow showing the translated strings in the user language, when it’s different from the wiki content language. This may be particularly relevant for multilingual sites such as Commons, Wikidata, Meta, and mediawiki.org.
MediaWiki’s usual fallback language chains must be used when a translation is not available. For example, if a message is not translated into Quechua or Guarani, it will be shown in Spanish, if it’s not translated into Bashkir or Chuvash, it will be shown in Russian, and so on. The ultimate fallback language is English, so if this message is not translated into Spanish or Russian, it will be shown in English.
Message keys
[edit]Messages should be represented as keys, similarly to how it is done in MediaWiki core, extensions, and tools.
Writing translatable strings will probably be the largest change in the template development process that template maintainers will have to get used to. Hard-coded strings will have to be separated and moved to messages organized by key. It must be made as easy as possible not only for the translators, but also for the template maintainers. Otherwise, they won’t actually do it, and the feature will be effectively rejected.
To easily make keys globally unique, it’s probably OK to automatically include the global template name in the message key.
Transitioning tools
[edit]A tool should be developed that will help the transitioning of a template or a module to central storage. It can do the following steps:
- Export a template from a local wiki and import it into the global wiki.
- Export all the templates that are used by this template (cascading).
- Identify the human-readable strings, convert them to a list with keys, and replace them with keys in the template’s source code.
- Import the template’s documentation page and TemplateData.
- Import the necessary CSS pages.
In most cases, this automatic process probably cannot create a fully usable and robust template or module, but it can help begin the transitioning process.
Organizing messages
[edit]The Translate extension organizes messages by groups, also known as “projects”, which can be further organized by aggregate groups. For example, Article Placeholder, Score, and Poem are all groups that represent the corresponding MediaWiki extensions, and they are all included in the “Extensions used by Wikimedia - Advanced” aggregate group, along with many other extensions.
Projects that represent MediaWiki extensions are configured in YAML files in the translatewiki repository and shown in the Translate user interface in a project selector, also known as “message group selector”.
Since there are many more templates than extensions, some modifications may be needed in the way the Translate extension handles message groups to accommodate templates translation.
Each template should be a message group. Closely related templates should be grouped in an aggregate message group. They can be similar to the categories in which they are stored, and in fact, the categories may even be reused. Editing files in a Git repository to organize these message groups is probably not desirable, because it’s too complex and slow.
It would be nice to show group and template names as localized in the selector, but it’s also OK if they are shown in English. If it’s good enough for extension localizers, it’s good enough for template localizers, too.
Templates must be shown as message groups on the Translate extension’s Language statistics special page (Special:LanguageStats). This will help localizers find what templates need translation. This should be generally similar to all message groups, but there are some special considerations for templates:
- There will be thousands of templates, so it will be nice if the table’s design corresponds to this somehow.
- The table should show on how many pages is each template transcluded and make it possible to order the rows by this number, to help localizers prioritize what is more important to translate.
Finding how to translate a template
[edit]Every template description page needs to have a direct link to translating it to the user’s language.
Some templates use Wikidata labels as part of their UI instead of hard-coding strings. This is done at the moment in Wikidata Infobox on Commons, Infotaula persona (Infobox person) in the Catalan Wikipedia, and in several other templates. These labels and values can be localized in Wikidata itself. Such usage cannot cover all the needs of template localization, but it is legitimate and useful for particular purposes. As long as this is properly described in the template documentation, this can continue to be used, and probably doesn’t need special infrastructure adaptations. (Perhaps the translation of the relevant labels and values can be somehow integrated into the Translate interface for localizing the template, but this is optional.)
Message parameters and magic words
[edit]In core MediaWiki and extensions, many messages have parameters, sometimes also known as “placeholders”. They are named $1, $2, etc., and they are filled in run time. Parameters are particularly important for making messages robustly translatable because different languages have different word order.
Something like this is needed in templates, too, although it is possible that the form does not have to be $1, $2, but template-like parameters with triple curly braces ({{{}}}). This is to be decided according to considerations of parsing and localization convenience.
The magic words PLURAL, GENDER, and GRAMMAR must be supported in template messages as in MediaWiki messages.
Message documentation
[edit]In core MediaWiki and extensions, every translatable message is documented for the convenience of developers and translators. The documentation may include information about where the message appears, what the $1, $2, etc. parameters are, whether the word is a verb or an adjective, etc. This documentation is stored as pseudo-language with the code qqq.
Such documentation will be useful for template translation, too. How it is stored is a question of technical architecture. Perhaps it can be combined with TemplateData, perhaps it can be stored as a qqq language, and perhaps it can be something else.
Source language
[edit]Templates will be imported to the global storage not only from English-language projects, but from wikis in many languages. More than ever, the localization tools must support translation from any language and not only from English.
Fuzzying
[edit]In core MediaWiki and extensions and in translatable pages, if the source message in English changes, the message is automatically marked as outdated or “fuzzy”. The existing translations continue to work, but are shown to translators as needing an update. (The translation manager can also mark a message as not needing fuzzying.)
A similar mechanism will be needed for template localization. However, since it would be nice not to force English as the source language, there should be more ways to mark messages as fuzzy.
Localization considerations for modules
[edit]Lua modules can load and parse translatable MediaWiki strings, but there is no defined way for storing these strings for Lua modules that are maintained as wiki pages. It is possible to package Lua modules as parts of extensions, and then they are able to load messages from the extensions’ i18/*.json files, but this is done in very few extensions at the moment. Rewriting templates in Lua may be a more robust solution from the engineering point of view, but Lua will not necessarily be embraced by all existing template maintainers, and their cooperation will be crucial to the project’s success, so this cannot be done to all templates.
Some very internal, technical modules that are commonly used, rarely changed, and don’t require internationalization can probably be painlessly moved into the Scribunto extension itself. Some examples are No globals and Arguments.
Localizing the template name
[edit]The template may have a different name in every language, but it must be directly connected to the central storage.
Global templates and modules are supposed to be immediately usable in all wikis without any extra steps, so it must be possible to transclude a global template in a local wiki page using its global name. The cross-wiki editors community shall decide what will be the policy for these global names.
Similarly to parameter names, templates may have different names in different languages, and this must be preserved. There must be a structured way to translate template names. Perhaps Wikidata sitelinks can play a role, but not necessarily.
If this is not done, editors will either avoid global templates, or wrap the global template in a local template with a translated name, and this will probably cause the template to lose the connection to the global entity. This is not desirable and misses the whole point of the project.
Template names must only be translated to languages that can be content languages of wikis. Translation to Formal German or British English are probably unnecessary. There may be a way to have aliases or redirects. Language variants, for example for Serbian and Chinese, must be supported according to these languages’ needs.
If a local template exists in a wiki and has the same name as a localized name of a global template, the local template will be used. This is similar to how local files with identical names override global files on Commons, and how local messages in the MediaWiki space override the localization coming from the code.
Lua module names are often localized, too. Their names can be localized for direct invoking from wiki pages, but since code usually uses English-like identifiers, the internal global names should probably be preferred for use in the code itself, for example in require statements.
Localizing parameters
[edit]Localizing parameter names
[edit]Parameter names are different in every language. They are usually based on words in each language, so it’s important for editing the transclusion in wiki syntax conveniently.
Ideally, the global template should have generic internal parameter names that have translations to different languages. This is somewhat similar to Wikidata property name labels, but it can be simpler: since English is a lingua franca for software developers and templates are a kind of software, it’s probably OK to have English as the default source language rather than generic numbers as it is in Wikidata.
These generic parameter names will be the common default names. They will work in wikis in all languages. The localized names will work in the wikis that has that language as its content language.
These translations of parameter names must be validated:
- they must not include invalid characters
- they must not be repeated within one template in one language
- Anything else?
The actual process of translating the parameter names may be different from translating user interface strings. These names have technical constraints that must be enforced. They must also remain stable because changing a parameter name will break existing transclusions, so there should be some safeguards against changing them too often.
Automatic parameter translation
[edit]If all the localized template and parameter names are stored centrally, it will be possible to have a simple service that gets a valid template call with parameters, a source language name, and a target language name, and outputs a localized template call. For example:
Input:
{
sourcewiki: "enwiki",
targetwiki: "frwiki",
template: "{{Infobox writer|name=Ľudovít Štúr|birth_date=1815-10-28}}"
}
Output:
{
template: "{{Infobox Écrivain|nom=Ľudovít Štúr|date de naissance=1815-10-28}}"
}
In Content Translation, this will be the primary way to adapt templates. Unlike the current template adaptation in Content Translation, this will be precise and complete, rather than based on guesses.
In visual editing and in 2017-style wiki syntax editing, simply copying and pasting a template from wiki in another language will do the parameter translation automatically.
For plain wiki syntax editing, there should be a simple way to operate this service, for example a special page or a dialog box where an editor can paste the template and the source language, and get the template with translated parameters.
In both cases, only the names of the template and the parameters will be translated. Translating the parameter values is discussed separately.
Nameless parameters
[edit]Nameless numbered parameters must continue working, of course.
A decision is needed about how will their names be localized.
Translating parameter values
[edit]In addition to making the templates’ functionality and design shared, some thought must be given to making the template parameter values shared, as well as not shared.
Some parameter values are the same in all languages by their nature. Some examples include an IPA pronunciation of a place’s native name (e.g. [dɛn ˈɦaːx] for The Hague), the year of foundation of a city, the chemical formula of a compound, etc. At least some of these should probably be stored in Wikidata and easily loadable in a template.
Some parameter values have to be translated or transliterated, for example people’s names, translations of country mottos, etc.
Global templates must make this possible, but in practice, these things are still often copied across wikis, and this must be taken into account as well.
Some parameter values can be reliably and predictably converted automatically, and the global templates infrastructure must support this. For example, number formats and digit characters are often different in Burmese, in languages of India, and in some other languages, but they can be reliably converted using simple software.
Valid and functional parameter values must be usable in multiple languages and must not be language-specific. For example, using “yes” and “no” as boolean values is too English-centric. This probably doesn’t require changes in the infrastructure, but mostly an agreement in the cross-wiki template development community on good practices for adaptation to all languages.
Text direction
[edit]Templates must adapt themselves to the text direction (ltr / rtl) of the wiki in which they are displayed.
It must be convenient to write a template in a direction-neutral way, with as little explicit right and left alignment as possible.
Bots
[edit]Many templates in many wikis are regularly edited by bots. This capability must be preserved.
This is not supposed to require any changes in the software infrastructure, but it is mentioned here for completeness because it’s an important use case.
Transitioning the templates from the large wikis to central storage
[edit]The most popular source language for translation in Content Translation is English, by far. After it, come Spanish, Russian, French, German, Catalan, Ukrainian, Italian, Chinese, and Portuguese. Because of this, it makes sense that the common templates in the editions of Wikipedia in these most common languages, especially those in English, are the ones that are the most important to make global for the benefit of all other languages.
Somewhat paradoxically, however, the editors in these largest languages are also the least interested in making them global:
- The templates already work well for them, and most people there don’t directly care about the convenience of translation to other languages.
- Rewriting the templates so that the strings will be translatable may be time-consuming and may force them to learn some new template maintenance skills.
- Making the templates suddenly used by many more projects may make it harder to achieve consensus about making future changes in how the templates work.
- Editors from different major wikis will have to work on reaching consensus about merging some templates with similar functionality that already exist on their sites.
This is more of a consideration of practicality and community relations than a consideration of engineering, but it must be taken into account when making technical architectural decisions. Without doing proper preparation in this area, the whole project will fail.
As long as there are important common templates that are not global, Content Translation and other software that handles templates from different wikis in any way, will have to keep supporting them. If the infrastructure for global templates is created, and migration of existing templates proceeds in a good pace, developers may consider stopping developing and some day deprecating the code for non-global template adaptation.
The pace of migrating templates from large wikis to the central repository can be one of the success metrics for the project.
It must be possible to use templates completely in both wiki syntax and in visual editing
[edit]It’s obvious, but should be mentioned anyway: Wiki syntax editing is not going away soon, and it must be possible to keep editing template transclusions in pages as it is done now. This must not become more complicated.
However, Visual Editor is increasingly embraced by both experienced and new editors, so every feature of how templates work must work well in both visual and wiki syntax editing.
Other features related to templates
[edit]There are some features that deal with templates in core MediaWiki and its extensions. All of them must continue working, and may need updating for the global templates age.
Core MediaWiki
[edit]There should be on-wiki tools for showing at least basic analysis of templates’ and modules’ usage on pages: the number of transclusions and invocations grouped by wiki, and lists of pages that use the templates and the modules. The feature that shows which templates does a page transclude while it’s being edited must continue working with global templates.
The “What links here” page must keep working, and remain useful for global transclusions.
TemplateData
[edit]- It is possible to translate template and parameter descriptions in TemplateData, and the translations are displayed in the user interface language in Visual Editor’s template insertion dialog. This is good and must be preserved. The translation interface could possibly be improved, but the beginning is good. Adding support for TemplateData in the Translate extension can be a solution for this, but there can also be other solutions.
- The “Suggested wikitext formatting” parameter (Inline, Block, Custom) must keep working. It must also be possible to customize them per wiki—some wikis may prefer to see a certain template written in wiki syntax as one line, and some may prefer multiple lines.
Citoid
[edit]- Citoid has to be configured on every wiki separately using JSON files, such as Citoid-template-type-map.json. In the global templates age, it must become possible to share these files, so that the “Cite” button would be available in all wikis and work identically everywhere by default. As with templates, there must be a way to override this default in each wiki where the community wants different behavior.
TemplateStyles
[edit]- There must be a possibility to write Template Styles pages in the same central repository as templates. The central style must be loaded by default, and it must be possible to override it locally.
TemplateSandbox
[edit]- The Template Sandbox must keep working.
- It must be possible to edit a template in the central repository and preview it in a page in the target wiki.
TemplateWizard
[edit]- The current system uses a wiki’s standard search to find templates. The results are presented in a list that might need to be changed to make the global or local status visible.
- TemplateWizard gets its information for templates from the TemplateData API, so as long as that keeps returning the same structure there shouldn’t be any issues, and i18n is already working.
Wikibase
[edit]- Wikidata can be used to bring in some parameter values from a central repository to the wiki. This is used productively in Wikipedia in several languages, among them French, Hebrew, Basque, Russian, Catalan, Estonian, and some others, as well as in Commons, although the actual implementation may differ. This must continue working, of course. Unifying the way in which it is done across different wikis may become one of this project’s most significant impact areas.
- It may also make it much easier to implement Wikidata Bridge, the project to allow editing template values from within wikis. The modifications to the templates themselves will have to be done only once in the global templates, and not in each wiki.
VisualEditor
[edit]- VisualEditor obviously needs to be able to insert both global and local templates.
- VisualEditor shows a link to the template description page in the template editing dialog. This link should point directly to the global template when it is used.
Development and deployment
[edit]Developing the infrastructure for global templates and modules is a large and complex project. It must be divided into manageable parts to get done. Roughly, the multiple parts of this project should be developed in the following sequence:
- Translatable modules (in development): Before making the modules shareable across wikis, the framework for their internationalization and localization should be developed. This will be immediately useful to modules on wikis that are already multilingual, most notably Commons and Wikidata. Some of them are translated using Lua arrays or tricks with translatable pages, but this is not a full-fledged localization system.
- Global modules: Modules become shareable across wikis. This should happen before making templates shareable because the modules’ infrastructure is less coupled to core MediaWiki.
- Translatable templates: This is similar to Translatable modules above, and can reuse much of the same framework, but it will also need the capability to translate the names of the template itself and its parameters and some other features. See the sections on localization in the specification.
- Global templates: Complete the project with making the templates global.
The development of more advanced features, such as making templates semantic, can and should come later, after they are shareable. If they become semantic before they are shareable, the code that describes them semantically will be forked on different wikis, like the templates themselves, which will make code reuse even harder than it is today.
Access to global templates and modules will be available from all the Wikimedia wikis. This includes editions of Wikipedia, Wiktionary, Wikivoyage, etc. in all the languages, as well as Commons, Wikidata, Meta, mediawiki.org, Wikitech, etc., as well as test wikis (test.wikipedia.org, etc.) This is similar to how images on Commons are available on all the wikis. Even though the global templates and modules will be available to the wikis, the wikis won’t be obliged to use them.
Making templates easily reusable on non-Wikimedia projects may also be desirable. Even though it doesn’t directly benefit Wikimedia projects, it may make sense to consider making templates easily reusable not only across Wikimedia projects, but also on other MediaWiki sites. Doing this will probably require some more work, but it may contribute to better modularization, and this may eventually benefit Wikimedia projects, too. This is comparable to the capability of direct embedding of images from Commons on non-Wikimedia websites. {{🌎🌍🌏}}
Imagine a world
[edit]Imagine a world in which every single human being can freely share in the sum of all knowledge and it’s actually a really easy thing to do because templates are global:
| Task | With the current template system | With global templates |
|---|---|---|
| Insert an infobox using the Visual Editor |
Note that you will have to find the template name for each wiki separately, and in some wikis it will not work at all. |
Note that the above will work the same way in every wiki and in every language, unless specifically overridden. |
| Insert an infobox using the wiki syntax editor |
|
If you want, you can still do all the same things as with the current template system.
But you will also have these additional optional features:
|
| Use a nice template that you saw in a wiki in English, French, Russian, or Spanish in the wiki in your language |
|
If the template is in the global repository, then:
That’s it, the template is fully usable in your language. (And yes, this also has to be repeated for every language, but the same is true for MediaWiki extensions.) |
| Use the new features of a nice template that you once exported into your language from a wiki in English, French, Russian, or another language | Three options:
|
Just translate the new strings. All the new features are already available in all wikis. |
| Implement Wikidata Bridge (editing Wikidata’s data directly from infoboxes) | This is a project in a very early stage of design. However, it can probably be said already that with the current technology, implementing it will probably require changing every infobox template in every wiki according to instructions that will be published by Wikidata developers. | The infoboxes’ template code will only have to be changed once, and it will work for all the wikis. |
| Translate an article from English, French, or Russian into your language using Content Translation |
In this scenario, steps 4 and 7 may take more time than step 3. |
In this scenario, step 3 takes most of the time, steps 4 and 7 are very short, and the total time is shorter. |
| Copy a useful citation of an academic article from the Danish Wikipedia to the German Wikipedia (or between any other two languages) |
|
In Visual Editor: Copy and paste without having to wonder whether it will work
In the wiki syntax editor:
(In the 2017 wiki syntax editor, the template conversion tool can probably be integrated into the pasting action.) |
| Have a table of the latest results of the ongoing Tour de France in the Wikipedia your language
This scenario is based on an actual module, Cycling race. It is not used in the English Wikipedia because of disagreements about the design of the tables, but the community appears to be open about adopting it in the future if necessary changes are made. |
|
Practically all the languages get the whole table for free, without doing anything. All the work is done by the people who create the templates and the modules, once for everybody, and by the people who follow the events and update the results. A cross-wiki platform for developing and localizing the module and the template may facilitate the development of a design that is acceptable in all languages. |
| Start writing in a new wiki after the domain is created | After the content is imported from Incubator, there are no templates for infoboxes, references, userboxes, discussion management, marking pages for merging or deletion, etc.
Start copying these templates one by one, or create your own. Nice people from other wikis may help you, but they probably do not know your language, so you will have to manually translate all the strings (as in the scenario called “Use a nice template that you saw in a wiki in English, French, Russian, or Spanish in the wiki in your language”). You can also just give up and only write text, but then you will get fewer features: no infoboxes, no formatted citations, no deletion and merging workflows, etc. |
All those templates are already available. Just translate the strings in a translatewiki-like interface. |
| Use a navigation box while reading an article on a mobile phone | Impossible.
Navigation boxes are so difficult to adapt to mobile screens, and so different from each other across wikis, that the software just hides them. (And yes, there is demand for it.) |
Possible.
Since the templates’ infrastructure is shared across languages, the different language communities can work with each other and with the developers of the mobile reading and editing interface and make them responsive. |
| Have a beautifully designed and mobile-friendly main page with regularly updating news, featured articles and images, etc. | Option 1: Find volunteers who know your language, as well as HTML and wiki syntax (especially tables and templates) very well and who have time to edit the main page every day. This is done separately and using different methods in every wiki. This happens in the top 70 wikis or so: English, Russian, French, Spanish, etc.
Option 2: If you can’t find anyone who knows your language and can also maintain the HTML code of the main page every day, copy the code from the main page from English or French and ask volunteers from other wikis, who don’t know your language, but know wiki syntax, to replace the picture of the day occasionally. Since they don’t know the language, they cannot maintain the text regularly, and you cannot do it because you are afraid of breaking the HTML code, so you get stuck with the same featured article on the main page for months or years. This happens in some of the smaller languages. Option 3: Give up and have a static main page, which most likely doesn’t work well on mobile devices, even though the main page is usually the top viewed page in a wiki. This happens in a lot of wikis, even if there’s other editing activity. |
Option 1: If you have people with the needed skills and time to maintain a manually crafted custom main page in your language, and you want to keep things that way, you can do it.
Option 2: If you don’t have the people who can do this manual work or if your community is fine with having a main page that looks similar to the one in other languages, put a simple centrally maintained template on your main page. Replace only the text of the changing items in an easy form, without having to deal with wiki syntax, tables, or HTML. The process is the same for most wikis, so the same templates and bots can be used by everyone who is interested. |
| Analyze how articles are sorted by WikiProjects for a research about Wikipedia |
|
|
(Note: The “With global templates” column assumes that the infrastructure is deployed in all Wikimedia wikis, and that the most often used templates are moved to the central infrastructure.) {{🌎🌍🌏}}
Status
[edit]This section is about the general status of the project. For details about the latest developments, as well as a brief history of past efforts, see the page Global templates/Status.
As noted above, as of December 2020, this page is only a big idea, and not a commitment to implement a project.
Similar ideas were suggested in the past. The oldest known suggestion to make templates reusable across wikis was raised in December 2004 in Bugzilla: Interwiki templates (task T3126). Several other similar ideas were raised later, for example Phabricator task T6547. In February 2017 a similar proposal called Global-Wiki was closed as “consensus”. Some of its components were implemented, such as global preferences, but not the templates.
The wish “Central global repository for templates, gadgets and Lua modules” was voted #3 at the Community Wishlist Survey 2015 and “Global gadgets” was voted #1 in Community Wishlist Survey 2016. Despite the community support, neither was implemented, because they weren’t appropriate for the Community Tech team, and they weren’t transferred to another team either.
The Platform Evolution project (2018) indicated some intentions to have support for global templates in the future. The page Platform Evolution/Recommendations discusses ideas for updating content modularity, and says:
- ... “boxes” are an ideal focus area for creating modularity. They represent self contained features and also an opportunity to enable equitable sharing of user features across projects and languages be establishing a cross-project service to share templates. This project will also force us to consider how to handle content layout and structure separately from composable pieces content.
The closely related page Platform Evolution/Goals lists this as one of the goals:
- Increase equity and power of contribution tools. We want to support the contribution of more content types of content, including media, in more interactive ways and across all projects. This means making some existing tools - like templates - available for consistent reuse across all projects and languages. It also means improving translation tools to remove silos of content. Finally, we also want to make it easy for contributors to create new cross-project, localizable content tools.
Abstract Wikipedia, which was approved by the WMF in July 2020 as a new project to be developed in the near future, includes “a cross-wiki repository to share templates and modules between the WMF projects” inside its Wikifunctions component as one of its major goals (Wikifunctions was previously known as “Wikilambda”).
The Translatable modules initiative was launched in September 2020. It addresses a part of this proposal.
In Community Wishlist Survey 2021, the wish titled “Templates translation” received the highest number of votes. It was also the wish with the highest number of votes in the history of the survey to date (December 2020). This wish closely corresponds with some parts of this proposal, especially Automatic parameter translation.
For other examples of the relationship between this Global templates proposal and various strategic plans in the wider Wikimedia community, see the page Global templates/Relationship to strategy.
There is no complete technical plan for implementing full-featured cross-wiki sharing of modules and templates. This page is an attempt to propose such a plan on a product level and listen to feedback from editors. {{🌎🌍🌏}}
Useful links
[edit]Some relevant pages that discuss similar topics:
- Platform Evolution/Goals
- Platform Evolution/Recommendations
- Multilingual Templates and Modules - An attempt to implement a similar feature using bots
- m:Community Wishlist Survey 2015/Results - Central Global Repository for Templates, Lua modules, and Gadgets came in as #3 in the Community Wishlist vote. Listed as “In development - Parsing team”, but not actually done.
- m:Which templates should be global? - an informal list made by various editors
- Requests for comment/Shadow namespaces - a dormant RFC about one proposal for a technical implementation of such an infrastructure
- Manual:$wgEnableScaryTranscluding - an existing rudimentary mechanism for transcluding content across projects. Considered inefficient and insecure, and disabled on Wikimedia projects.
- m:Global-Wiki - a similar proposal, with a wider scope. It was open for discussion for several years, and closed as “consensus”. Some things in it were implemented, such as global user pages and preferences, but it also includes global templates, which are not yet done.
- m:Abstract Wikipedia - A bigger proposal, which includes a global module and template repository (also known as Wikifunctions, and previously known as “Wikilambda”).
- m:Community Wishlist Survey 2021/Translation/Templates translation - A community wish that corresponds to some parts of this proposal, especially Automatic parameter translation. {{🌎🌍🌏}}