User:Adamw/Drafts/Component templates RFC

This document describes a suggested enhancement to wikitext templates.  It began as an exploration for the WMDE Technical Wishes, but has drifted into “rogue RFC” waters.

Proposal

Introduce a new format for single-file templates, we call it “component” templates here, that flexibly encapsulates multiple heterogeneous blocks of different types, such as wikitext templates, Lua scripts, front-end ECMAScript, template docs, TemplateData, i18n messages, and stylesheets.  This integrated format is inspired by the recent Web Components standard and by trends towards single-file components.

A component template may name its child elements, which makes them addressable.  For example, a script could render multiple subtemplates independently, looking each up by name.

Component templates are backwards-compatible with discrete pages for each content type, and the two paradigms can be safely mixed.  In fact, discrete pages are one way to override component fragments.

Motivation

To quickly paint a backdrop for Wikipedia’s template support, it was added in small steps, beginning with transclusion in 2003.  This was a time before standardized languages for template processing (PHP itself was considered such a language, for example), this was a Wild West in which each company would invent their own frameworks.  The architectural mantra “model-view-controller” was dominant, and programs were often organized into “layers”.  The Template namespace is one such layer, and was originally meant to hold pure presentation.  By 2006 the “#if” statement was added to the language, bringing with it the burden of Turing-completeness.  An attempt was made to divert this flow of logic into templates by introducing Lua, a full programming language callable from the original templating system.

The main problem is that the dependencies are pointing the wrong way: in MVC the controller is supposed to manipulate models, then model updates propagate to the view.  A presentation template shouldn’t have to “know” anything requiring advanced switching logic.  If we flip this dependency and wikitext templates are no longer responsible for logic and control flow, extracting that logic will simplify the presentation.

Moving to a single-file format addresses a second major shortcoming, that the template “layer” obscures important structural facts, that templates exist to accomplish a limited behavior, and that they exist in a tightly coupled matrix of supporting content types.  The boundaries between components cut across these different types.  Collecting related fragments together gives us logical locality, it becomes easier to reason about a feature.

Putting these two points together, scripted logic keeps control and can compose the component’s content in any way desired.

A single-file format should lower the barriers to reusable “global templates”, because components are structurally compact and portable.

The “lang” attribute adds the flexibility to experiment with a variety of templating and scripting languages serving the same purpose, by supporting transparent interchangeability.

Encapsulation and embedding of the different supporting content types (, <templatedata>, Lua logic, docs) is already taking place, so this is partially consistent with existing momentum for change.

Implementation

The prototype can be written as a tag extension.  Wikitext enclosed in “<component>” tags will be treated specially, and its contents will be interpreted by the extension.

Status: Successfully implemented the <component> tag and local template transclusion, can run the examples in this document.

Proof of concept: https://gitlab.com/adamwight/ComponentTemplate

Persistence

Namespace

Components will live in the Template namespace, or in MediaWiki if using privileged tags.

New tags

The page content is a root <component> element.  Child blocks like <template>, <templatedata> and <script> build up the content and functionality.

Each child element may optionally have a name.  This allows for multiple wikitext or style snippets, individually addressable and overridable.  Various flavors such as wikitext vs. mustache, or less vs. css can be chosen through the “lang” attribute.

Atomicity

Atomicity is a nice-to-have which legacy templates are lacking.

It should be possible to avoid a broken, intermediate component in the event that multiple fragments need to be updated simultaneously.  For example, a stylesheet change corresponding to a newly added element.  This requires that component source code and cached object code can be updated atomically rather than each fragment updating in sequence.

Transclusion syntax

Component transclusions have the same syntax as normal templates: double curly braces, pipe characters, and string values:

{{ABC|1=alpha|legend=Alphabet}}

Component templates are called like legacy templates. They can also be transcluded directly from Lua,

frame:expandTemplate( ‘ABC’, args = { 1 = ‘alpha’, legend = ‘Alphabet’ } )

Lua to call a component, same as for legacy templates.

It would be possible to wrap Lua’s “require”, so that a component can be loaded like a module, but still take along its payload of presentation, etc.

Evaluation

Parameter binding

A component template receives its arguments as a {string:string} map bound to the script’s frame.args, just like legacy templates.

Entrypoint

Compilation of a component goes like,

• Evaluate the Lua script if present (see below).
• If no script tag is present, then the first template element is evaluated and returned.

Let’s create a new convention for Lua scripts when used in a component template context: if the export table includes the function “__call”, that will be called with the template arguments.  This simplifies script integration and reduces the boilerplate in each call.  “{{#invoke:ABC|main|1=arg1}}” becomes “{{ABC|1=arg1}}”.  If the “__call” function is available on a Lua script’s returned interface, this becomes the entrypoint and receives all template parameters.

As a courtesy, when the Lua script is missing a __call function, we assume that the script is written for #invoke and inject a compatibility shim which behaves as below.  It takes the first template parameter, uses it as a function name, and shifts the remaining parameters.  In the above example, we could call the template like “{{ABC|main|1=arg1}}”.  This is how the shim is implemented:

function p.__call(self, func_name, ...)

return self[func_name](...)

end

Implicit compatibility shim dispatches to a named function.

Static elements

Stylesheets, client-side scripts, embedded images, and other static elements are carried over to the final output, and deduplicated rather than appearing multiple times.

Lua can control whether static elements are output, addressing them by name.

Security

There will have to be a distinction made between types of fragment, so that permissions can be differentially applied to each.  For example, template doc subpages should be left open to all editors even when the template itself must be protected (because popular templates are highly-valuable to vandals).

One alternative is to disallow certain tags in the Template namespace, but allow in the admin-only MediaWiki namespace.  The disallowed tags might include frontend scripts and stylesheets.

Another more complex alternative would be to block page save only if insecure fragment types are modified, in any namespace.

Caching

For the prototype we can rely on normal page caching.  If transpilation steps cause latency, we can cache the object code.

Customization

Many wikis will want to customize a template, its presentation, and translations.  Forking a template is simple and effective, but causes problems in the long-term.  We would like to do better with components.

Any named element can be overridden by creating a subpage with that name.  To override a subtemplate in Template:Foo called “name”, create Template:Foo/name .  To protect an element from customization (“closed for modification” / “final” / “private”), either don’t give it a name, or use an ad-hoc indicator such as a leading underscore: “_name”.

If Template:Foo calls {{name}}, the lookup order will be:

• First, check for a subpage override: “Template:Foo/name”.
• Next, search for named siblings within the component: “Template:Foo#name”.
• Finally, look for a normal template: “Template:name”. [may be undesirable to mix local and global template lookup]

Migration

A legacy wikitext template can be turned into a component by wrapping it in “<component>” and “<template>” tags.  Style blocks, subtemplates, and additional elements can be added incrementally.

Examples

These snippets can be evaluated using the proof-of-concept ComponentTemplate extension.

Use case: Move logic out of templates, and presentation out of code

Even something as simple as parameter fallbacks have proven unwieldy with wikitext templates:

Author name: {{{penname | {{{pseudonym | {{{name}}}}}}}}}}}

Template:Infobox writer—tainted by logic One might try to shift the complexity of parameter aliasing to Lua, where we expand another template or perform the template’s presentation work using string concatenation.  For example,

local p = {}

function p.clean(frame)

    local args = mw.getCurrentFrame():getParent().args

   local author = args.penname or

       args.pseudonym or args.name

   return “Author name: “ .. author

end

return p

Module:Infobox writer—tainted by presentation

But moving the concatenation into code also goes against the isolation of logic, this is a presentation detail and should be left in a template.

Here’s the same logic as a component, but with the dependency reversed so that the script expands a template.

<component>

<template language="wikitext">

Author name: {{{author}}}

</template>

<script language="lua">

local p = {}

function p.__call(frame)

   local args = frame.args

   return mw.ext.ComponentTemplate.expandLocalTemplate(

       "Template:Infobox writer as component", {

           author = args.penname or args.pseudonym or args.name

       })

end

return p

</script>

</component>

Template:Infobox writer—as component

Use case: Simplify Lua integration; move templating out of code

Lua scripts have so far been in the Module namespace, and the Lua invocation is almost always hidden in a dedicated, wikitext, Template page.  Using templates from within Lua is rarely done, probably because of the danger that templates will change and break tight coupling.  This friction in both directions could be reduced if the Lua and its wikitext template could be packaged together.

For example, Lua modules are often wrapped in a shim template like this one:

{{#invoke:City rainfall|main
|city_name = <nowiki>{{{city}}}}}

<noinclude>

<templatedata>
{
   “description”: “Given a city name, display a rain gauge with the historical average rainfall and decorations appropriate for the current weather.”,
   "params": {
       "city_name": {
           "label-en": "City name",
           “description-en”: “Nearest city with recorded weather.”,
           "example-en": "bar.",
           "type": "string",
           "required": true
       }
   }
}
</templatedata>

Template:City rainfall—Non-component implementation

Currently, many Lua modules generate wikitext and HTML directly using the `mw.html` library and string concatenation.

<script lang=”lua”>
local weatherLookup = require('Module:Weather lookup').main

local function renderRow(location)
   return [[
|-
| class=”field-name”   
| ]] .. location.city_name .. ‘\n’ .. [[
| ]] .. location.average_rainfall .. ‘\n’
end

local function renderTable(weatherReport)
local tableRows = {}

for location in weatherReport.locations do
       table.insert(tableRows, renderRow(location))
end

return [[
{|
! {{int:city-label}}
! {{int:average-rainfall-label}} (]] ..
weatherReport.unit .. ‘)\
‘ .. tableRows .. ‘|}’
end

local p = {}

function p.__call = function(frame)
    return renderTable(
         weatherLookup{
            city = frame.args.city,
           fields = { ‘rainfall’ }
        }
    )
end

return p

</script>

Module:City rainfall—Non-component implementation

As a component, the pieces could be bundled together as one page, and templates fully extracted from the script.  The component renderer can detect whether this is a transclusion or template page view, and emulate the <noinclude> / <includeonly> logic implicitly.

<component>
<templatedata>
{
    “description”: “Given a city name, display a rain gauge with the historical average rainfall and decorations appropriate for the current weather.”,
    "params": {
        "city_name": {
            "label-en": "City name",
             “description-en”: “Nearest city with recorded weather.”,
            "example-en": "bar.",
            "type": "string",
            "required": true
        }
    }
}
</templatedata>
 
<templatedoc lang=”wikitext”>
{{Wikitext}}, <b>HTML</b>, or whatever, documenting your template.
</templatedoc>
 
<template lang=”wikitext-template” name=”main-table”>
{|
! {{int:city-label}}
! {{int:average-rainfall-label}} ({{{unit}}})
{{{tableRows}}}
|}
</template>
 
<template lang=”wikitext-template” name=”table-row”>
|-
| class=”field-name” | {{{city_name}}}
| {{{average_rainfall}}}
</template>
 
<script lang=”lua”>
local weatherLookup = require('Module:Weather lookup').main
 
local function renderRow(frame, location)
    -- This is an exciting new function which loads
-- templates from the component itself.
    return frame:expandLocalTemplate(
        name = “table-row”, location
    )
end
 
local function renderTable(frame, weatherReport)
local tableRows = {}
for location in weatherReport.locations do
    table.insert(tableRows, renderRow(frame, location))
end
return frame:expandLocalTemplate(
         name = ‘main-table’,
        {
            “tableRows” = tableRows,
            “unit” = weatherReport.unit
        }
    )
end
 
local p = {}
function p.__call = function(frame)
     return renderTable(
         frame,
         weatherLookup{
             city = frame.args.city,
            fields = { ‘rainfall’ }
         }
     )
end
return p
</script>
 
</component>

Template:City rainfall—Component implementation

Use case: Frontend interaction technologies

The single-file design is extensible, all we need to do is match elements to a preprocessing handler. Additional element types might include an open choice of transpiled, front-end languages which enable interactivity and modern tooling. See the “Future Directions” section. Platforms we could support include React, Vue, and TypeScript. As an example, an interactive timeline might respond to clicks by highlighting the corresponding article sections:

<component> <template lang=”vue-template”> <div> <timeline-slider :interval=”1” :min=”1920” :max=”2020” v-model=”articleContentTimeline”> Timeline leading to the general strike. </timeline-slider> </div> </template> <style lang=”less” src=”timeline-slider-component/theme/material.css” /> <script lang=”ts”> import TimelineSlider from 'timeline-slider-component'; import 'timeline-slider-component/theme/material.css'; export default class TimelineSlider export default class CustomizedTimelineSlider extends Vue { private articleContentTimeline: number[]; ... } </script> </component>

Future template with transpiled, front-end scripts.

Strategic outcomes

We don’t know yet whether this feature will satisfy any of the templating challenges identified already.  This proposal is just one of many alternatives to evaluate.

Outputs

• Pushing templates below scripts in the dependency graph reduces their complexity.
• Larger granularity of encapsulation for user-authored wiki elements.
• Facilitates adoption of new script and template syntaxes.
• Reusability?
• Easier comprehension of templates, less work required to find all the pieces, lower barrier for the technical skills required, more accessible to more users.
• Easier to author as measured by what a single copy-and-paste will provide you.

TODO: Anticipated outputs.

TODO: Outcomes and how they are supported by outputs.

TODO: Impact and how it relates to existing goals.

Open questions

• Aligned initiative: mw:Platform Evolution/Recommendations#2. Create a service for rendering components from template content.  The WMF Core Platform team will be starting work on this in a few weeks (December-ish).
• Previous, related suggestion: HTML content templating.
  • Makes a good point, that dependencies could be abstracted as an URL, archived and cached on the server after each access.
  • TODO: makes lots of other good points, summarize them here.
• Should there be flavors of components which can emit unsandboxed HTML, dictionaries, …
• Maybe components live a new namespace?  But this leaves us the choice of either “{{}}” transclusion searching both the new and old namespace, or requiring an unusual syntax to transclude.  These both sound bad, and we’ve seen from experience that “{{#invoke” has been a barrier to using Lua directly.
  • Another option is to keep components in a non-main slot in the Template namespace.  There could be a slot for each component type, although multiple templates might break this organizational scheme.
• Declaring dependencies would be much nicer than discovering programmatically, because subtree pruning is a major obstacle to exporting the full set of dependencies to a template.  However: Web Components allows for dynamic loading.  And it’s much easier for new authors to have everything available.  We can simply recommend that requires are done unconditionally, to make static analysis possible.
• TODO: Unclear whether component encapsulation will help with sharing “global” components and modules.  The new idea we can offer is to wrap the entire functionality of inner components using monadic composition
  • I don’t see how to stay granular as we pack into a component.  Ideally we could override at the subtemplate level, as with i18n messages.  The asymmetry with e.g. exporting translations shows that storage assumptions might be wrong.
    • Granular overrides would be possible in Lua if an entire component could be treated as a table.  Or at the element level by a tag e.g. “<component src=...” that can do something clever with the inner component’s namespace.
  • One half-thought is that individual snippets such as a table row presentation might be included as template elements in a component, and wrapped in a calling component which can override component elements by name, etc.  Upstream templates might live in Global:Template:Foo, and serve as a fallback for Template:Foo, or if Template:Foo exists it can explicitly wrap the global.  Somehow.  Intentionally open languages like CSS provide a simpler approach in which the wrapping component can override simply by appending styles.  Subtemplates within a component would have to be overridden by name.  This is sort of an argument against having multiple templates.
  • FIXME: if we’re breaking Liskov substitution by having a localized template accept different (e.g., translated) parameters than the global template it shadows.  Calling this sort of template is ambiguous—which parameters to use?
• There might be good reasons to allow some or all components to use the “template calls code” organization, which could be implemented as a magic flag attribute such as “mainTemplate” on the template which causes evaluation to start with the template rather than the script’s entrypoint.
• Are component template parameters still wikitext?  Or plain text or HTML?  JSON dictionary?  How are they written when invoking a template?  Compatible with the existing named and positional, wikitext-value syntax?
• Similarly, does transclusion result in wikitext plus limited HTML?  Or full HTML?  Or a JSON dictionary that can be read into the calling function?  Should we support several flavors, e.g. a legacy style that handles wikitext, a future-compatible style, and a privileged mode where any HTML tags can be used…?  It’s up to the caller to interpret component output correctly, e.g. only call a structured output module from within Lua.
• Should we be strict about a component template having a single root element and no surrounding whitespace, or should we be more robust, trim spaces, allow multiple components, allow wikitext followed by a component, etc.?

Future directions

• Other scripting languages: This component structure may organically lead to interest in other technologies supported by Web Components.  For example, interactive elements using Vue or React in the browser.  Parameters could be bound to frontend elements by serializing to JSON.
• Components should be allowed to communicate with one another, what would this look like?
• Single file component standard: Follow webpack and other emerging standards.  For example: [ref].    Nesting seems suspicious, I’d like to think that the component has documentation and templatedata, rather than being specific to the script tag.
• SSR extensions: There’s enough industry investment server-side rendering of web components (during testing, or for runtime PayPal, Netflix) that we might see affordances for running on the server integrated into the standard.  We want to adhere to this or be sure we’re forwards-compatible.