Jump to content

Requests for comment/Future of magic links

From mediawiki.org
Tracked in Phabricator
Task T145604
Request for comment (RFC)
Future of magic links
Component General
Creation date
Author(s) Legoktm
Document status accepted
See Phabricator.


Background

[edit]

Magic links are a feature of MediaWiki core that create automatic links for 3 hardcoded external identifiers:

For the purposes of this RfC, we are not considering free external links (e.g. typing just https://www.example.org) to be a magic link.

Problem

[edit]

These are hardcoded, inflexible, un-localizable, and generally unexpected. If this feature were proposed today, it would be rejected in favor of using templates or interwiki links. There have been long standing requests to make them disable-able, move them to an extension, or remove them outright.

In many cases, local templates are preferable and more advanced than magic links. For example, on the English Wikipedia, Template:ISBN checks for invalid ISBNs and adds them to a tracking category for editors to fix up.

Proposal

[edit]

As of gerrit:309528 it is now possible to disable magic link functionality using $wgEnableMagicLinks . Our eventual goal should be to remove all magic link functionality from MediaWiki core. While moving them to an extension would allow us to remove them from core, it would probably entrench magic links even deeper in the parser as we would need a mechanism and hook system to support an extension to add magic links.

Instead, we should:

  • Yes Done Add a tracking category (via ParserOutput::addTrackingCategory()) to any page that uses a magic link (3 categories, one for each link type)
  • Add a {{ISBN:...}} parser function that replicates the functionality of the magic word (validation and linking to Special:Booksources).
    • It has also been suggested that we could use a "ISBN" virtual namespace that redirects to Special:Booksources
    • Discussion on phab:T148274 is still ongoing.
  • VisualEditor and other editing tools would continue to support ISBN/RFC/PMID "magic" links as they do, but convert it to a proper link client-side.
  • Yes Done RFC and PMID should be added to the Wikimedia interwiki map and the MediaWiki default one if they aren't already.
  • Yes Done In the MediaWiki 1.28 release, default magic links to being disabled. Wikimedia wikis would still have it enabled for now.
  • In progress In progress Encourage users to migrate to using the parser function and interwiki links or local templates (e.g. w:Template:ISBN), probably using bots and other assisted tools.
  • Remove magic link capabilities in MediaWiki 1.30 (1 year later, also LTS).
    • Update: this part was controversial, and will be re-evaluated based on the response to disabling by default.
    • We would retain ParserOptions::getMagicISBNLinks() and co. but they will always return false, in order to signal to extensions that the magic links are disabled.
    • Old revisions would no longer have autolinks, but that should not reduce the readability of the content as they are well known identifiers.
    • Move Special:Booksources and the ISBN parser function to an extension.

Comments

[edit]

What for would you do this? It will clutter all the projects with extra template calls and other extraneous crap. Rich Farmbrough 18:03, 27 December 2016 (UTC).[reply]

There's concern on enwiki (here, for example) about the loss of the ISBN and PMID magic links. There's also concern about the lack of discussion. The first most of us knew about this was when a bot operator submitted a request to remove the links. Can the disabling of the links be halted so that a wider discussion can be held first? SarahSV (talk) 19:05, 29 December 2016 (UTC)[reply]

  • Lets look at one "reason" un-localizable, that issue is just tagged as "won't fix" for "the absence of objections." I would have said that localisation is totally trivial, certainly as easy as localising a set of templates. Rich Farmbrough 23:16, 2 January 2017 (UTC).[reply]
Has there been any real discussion on the actual sites? This will have a huge impact. [It seems to me that this change goes against consensus? See Removal of ISBN magic links.] Jeblad (talk) 13:12, 15 April 2017 (UTC)[reply]
Yes, discussions are on-going. The English Wikipedia had an RfC and agreed to deprecated and eventually remove magic links. Legoktm (talk) 00:31, 12 July 2017 (UTC)[reply]
Have you asked the German speaking community? Please provide a link. --Eingangskontrolle (talk) 15:06, 27 March 2018 (UTC)[reply]
  • At first I thought this cleanup might be a good idea, then I started to wonder if magic links is just the kind of feature that might solve the currency converter problem. Now we write into articles what a specific currency might be, and how it might be converted to a local currency, but by using magic links for the short three-letter codes we could simply parse out the currency and add a currency converter without cluttering the wikitext. This is not that common, so perhaps it is acceptable to use a parser function in those cases. It would probably not go long before someone have created templates for all currencies, with no real additional functionality, if it is added as a parser function. I would prefer silent parsing of an inline magic link. [Just to be clear; I do not support this RfC.] Jeblad (talk) 13:18, 15 April 2017 (UTC)[reply]
  • I think this is a really, really bad idea. It will result in many, many, many future ISBN citings for which editors won't use the template and thus won't be linked anymore. I am strongly opposing this RfC. --Matthiasb (talk) 05:20, 23 May 2017 (UTC)[reply]
Please keep the magic links, why increase work load for editors? And why clutter the source text, so that it becomes less readable to the vast majority of non-coders (writing an encyclopedia) and potential new authors? Cheers, --Ghilt (talk) 11:00, 23 May 2017 (UTC)[reply]
  • Magic links are useful, templates are geeky. So the more templates, the less attractive the wikiverse will get for non-geeks. So like Matthiasb, Ghilt and Gretarsson. And this has to be discussed on at least 50-70 wiki-projects themself, not only here in some back room of geeks. Grüße vom Sänger ♫(Reden) 15:50, 23 May 2017 (UTC)[reply]
    In general editors should never have to deal with magic links directly, nor do they need to know they have to add templates themselves. For example, w:Template:Cite book and related automatically link ISBNs, so users don't see any difference. Users who have trouble with wikitext can use Citoid's citation generator which handles all templates for them. Legoktm (talk) 00:31, 12 July 2017 (UTC)[reply]
  • You propose to disable an extensively used and useful feature without any need, without any existing alternative and (therfore) without any improvement. CONTRA --Fano (talk) 16:03, 23 May 2017 (UTC)[reply]
    The intial motivation is that we're embarking upon cleaning up wikitext, standardizing it, and modernizing it (see the related tidy migration and Linter project). This effort is being done so we can improve tools for editors, and get rid of a lot of technical debt. Magic links have always been a weird part of wikitext, and developers have often been unhappy with the concept due to how magical it is ("explicit is better than implicit"). Alternatives have been proposed and currently the one everyone agrees on is using templates. I don't disagree that for some people this will be a regression in functionality, but overall and in the long run it will be an improvement in my opinion. Legoktm (talk) 00:31, 12 July 2017 (UTC)[reply]
  • I miss any reason for abandoning these widely used features. The claims above under the header "problem" seem ridiculous to me. Un-localizable is patently untrue, as localizing them is as easy as apple-pie. Hardcoded is exactly why they are preferable to templates, they need no user activity. And while they are inflexible, that too is exactly what they are supposed to be. They fulfill one purpose, and do so very well. Unexpected is the most laughable reasoning I have read so far on this wiki. These features have been coded for a reason, they shall and do link three well defined information patterns of utmost importance to well established resources. Some years ago you removed DOI from the magic words, and I still miss it. So my proposal would be to reintroduce DOI as magic word ASAP, and certainly keep the three existing ones. --H-stt (talk) 09:26, 24 May 2017 (UTC)[reply]
If you think something is "as easy as apple-pie" it's a shame you didn't submit a patch earlier to implement that functionality - however I doubt it's as easy as you think. And while you might see hardcoded/inflexible as a feature, many people see it as exactly the opposite. As for being unexpected, see phab:T70217, or w:Wikipedia:Village_pump_(technical)/Archive_8#Automatic_external_link, etc. AFAIK DOI has never been a magic word, and I certainly never removed it. It's still an interwiki link if that's what you meant. Legoktm (talk) 00:31, 12 July 2017 (UTC)[reply]
  • Comment from technical site: It will be very helpful to change the Magics to Templates, because of a better possibility to localize or search those contents by API or as database entry as Table Templatelinks including parsing key and value. The edit on article pages is not very more difficult, but we should use our de:Template:Zitation simply a little more to catch such entries. I only can see a helpful optimization and we should do this. Doc Taxon (talk)
  • The ISBN magic link has been in use several dozen million times, I guess. This RfC is, in comparison, absolutely nothing. I can't take this seriously the way it looks. Man77 (talk) 16:08, 25 May 2017 (UTC)[reply]
    Why do you think this RfC is "absolutely nothing"? Legoktm (talk) 00:31, 12 July 2017 (UTC)[reply]
    Because it ain't done where those links are used, just here with the geeks. Go to the portals in the wikis, where they are heavily used and make some dozen RfC there, and then come back here. Templates are far less useful for editors as plain text, that automagically gets transformed, templates shoo new users without much interest in using strange syntax away. Grüße vom Sänger ♫(Reden) 16:10, 12 July 2017 (UTC)[reply]
  • I am agnostic on the proposal itself, but advise that the recommended list of alternatives should include {{cite xxx}}, e.g., {{cite book|title=foo|isbn=bar}}.
  • The parts of the proposal that are shown as done are good to have (like putting in the ability of disable magic links), but I strongly oppose the idea of removing magic links in total. Even if all present ISBNs in all Wikimedia projects are migrated from templates to magic links, this change just adds yet another burden to editors and clutters page source. As per H-stt's opinion, I also support adding back DOI to magic links. If you don't like it being hardcoded, move it to an extension and make it customizable there. Removing it without considering the impact to (the future of) existing projects just does not make sense. --ネイ (talk) 15:55, 19 June 2017 (UTC)[reply]
  • A waste of editors' time, with no clear explanation. @Legoktm, the benefits mentioned were vague and the feedback was quite negative. Why did you press ahead? The decision-making process seems to have failed here. What is the correct venue to address the decision-making process? --Hobbes Goodyear (talk) 01:09, 25 June 2017 (UTC)[reply]
  • Support I assume this is used by the visual editor people? Typing [[ISBN/0-7475-3269-9]] or {{ISBN:0-7475-3269-9}} instead of ISBN 0-7475-3269-9 is not difficult. Power~enwiki (talk) 21:29, 4 July 2017 (UTC)[reply]
  • This is the first I'm learning of this. The wikis I run use magic links extensively. Using journal as an example, our citation format has been and will continue to be <ref name="Name">{{cite journal |title=Title |journal=Journal |author=Author |volume=Vol |issue=Iss |pages=pp |year=Year |doi=DOI |pmid=PMID |pmc=PMC}}</ref> with a references section at the bottom. I don't claim to fully understand what you all are proposing to do. All I know is that for now I just have to plug in a DOI, PMID, etc. into the citation template and it works. It sounds like putting those IDs into the template would no longer produce links in the results shown in the references section; we'd somehow have to add an additional template and thus make it more complicated. The wiki is complicated enough, particularly for new users. Against this change, for the same reasons as H-stt. Lostraven (talk) 18:56, 6 July 2017 (UTC)[reply]
    If you're using a standardized template, then this change affects you very little. Just have the template create the links directly (e.g. [[Special:BookSources/{{{isbn}}}|{{{isbn}}}]], and in the articles themselves nothing will change. Legoktm (talk) 00:33, 12 July 2017 (UTC)[reply]
  • Support - the Magic links functionality should be moved to an extension that Wikimedia projects can debate about in terms of future use. Magic Links as currently set up breaks my external wikis, such as https://gowiki.tamu.edu, and others that rely on creation of page titles based on PMIDs. I have not figured out how to patch this yet, which is preventing upgrades to more recent versions of MW (currently testing with 1.29) JimHu (talk) 20:48, 8 April 2018 (UTC)[reply]
  • Support with caveats: I have used MediaWiki's magic-link type functionality in quite a few private wikis: in many organisations, referring to "Job 1234" is an unambiguous reference to a given project; so I've made Job 1234 expand into [[Job 1234|Project Blah Blah]]. (Currently I do this by modifying the existing special-case code to recognise the tag and do a database lookup for the project name. While it is certainly ugly, for these groups it is extremely useful.) But those same wikis have RFC, PMID and ISBN disabled, as they are not meaningful for their user groups. So if we generalised it away into an extension, to ease Gig 1234 or Client 1234 that would be great. I truly understand these uses are not likely to help Wikipedia etc, but they enormously have helped the organisational wikis I've put in. Thus I oppose simple removal, as it would become much harder to do. Jonathan. 82.69.229.22 13:09, 27 April 2018 (UTC)[reply]

For your information, there is now Extension:MagicalLinkers. Nuno Tavares (talk) 13:09, 7 August 2018 (UTC)[reply]

  • Removing magic links seems more and more like the wrong action to me. It is advocated as an action solving a few claims; hardcoded, inflexible, un-localizable, and generally unexpected. The claim that they are hardcoded is about the current implementation. If the implementation is wrong, then fix it! The claim that they are un-localizable is also about the current implementation. Once more, if the current implementation is wrong, then fix it! Then a pretty vague claim that they are "generally unexpected". I believe that is a false statement. Something that has been on Wikipedia for so many years is not unexpected. It might be unwanted for some users, but it is not unexpected. (This is like an insurance company getting a claim about a tree unexpectedly growing up in front of a car.)
    This kind of functionality is infact very important as it annotate text without any user involvement. If we could generalize on the idea it could be described as a way to rewrite a pattern into a template call, which could then be redirected into a module and a gadget. A call on a three-letter currency code, followed by a number, could then be redirected to a currency-converter. Likewise a call on a value with a following unit could then be redirected to a unit-converter. A third is a converter between date and time in different timesones and different calendars. (There are several similar examples.) We can move these examples out in separate extensions, but the problem is general and should be solved in the core itself.
    A simple implementation is to use regular expressions to find the patterns, but this can be very heavy if users are allowed to add them. Still this is more or less whats done in the AbuseFilter extension.
    I'm not sure, but I wonder if the problem is that some types of natural language is closer to structured data, and in some cases the structured data is so close that we can utilize it as-is. In those cases it virtually makes no sense to wrap it in additional layers of formatting. In these cases the additional layers are just noise. ISBN codes are such structured data. Additional templates, parser functions, or whatever, are just noise. Jeblad (talk) 21:01, 18 December 2018 (UTC)[reply]
  • I am for this RFC, and I will explain why the #1 argument against this RFC is also the #1 reason for this RFC.
    Suppose I wrote "it's common knowledge that Newton discovered gravity", you would notice two things: 1) neither "Newton" nor "gravity" is linked to any article, and 2) no text is formatted as bold or italic. Why? Because given the context of the article, the author (I) don't feel there should be any link or formatting. Wikitext should be predicable. If the author doesn't explicitly link some text, there should be no link.
    The argument that "magic links are widely used" and "removing the functionality would mean more work on editors" aren't really valid. Why should RFC, PMIC, and ISBN be given special consideration over "Newton", "Obama", or "year 2000"? A new editor would probably be very confused why RFC 123 would create a link. Is it important enough to warrant a link? Maybe not. Is it more work to disable the link? Absolutely! The fact that magic links are everywhere could be simply that the links shouldn't be there in the first place and it's just too much work for editors to selectively disable individual magic links so only the important ones remains. In Wikipedia or any other wiki site, every link and every formatting should be a conscious effort in consideration of the context. Yes, initially it would be more work to convert all magic links to use templates, but then again, why should all such links be converted? Dan1wang (talk)
  • Strong oppose Such a severe cut can not be discussed here in the backyard. Open a voting in the local projects where this feature is heavily used if you want to remove it. Chaddy (talk) 03:04, 22 April 2022 (UTC)[reply]
  • Strong oppose Why should an editor avoid to link the ISBN number? You suggest, that every day hundreds of new articles have to be reedited just to get back the links. Dan1wang does not understand the difference between a prefix of a number with a certain information and just plain text.--Bahnmoeller (talk) 08:22, 7 July 2022 (UTC)[reply]
  • Actually the argument of unlocalizability is a no-brainer, sorry, since ISBN is just ISBN in each and every language of the world 'cause it is an international standard. It's the main purpose of international standards to deny localization. Likewise PMID. "Obama" is no international standard. The idea to use Template:ISBN instead conflicts with the long-standing trend to discourage in-line templates in the German wikipedia; for example de:Template:ISBN was rejected and deleted in the German WP in 2005 already. Citation templates won*t resolve it as in German Wikipedia citation template using isn't widely accepted. Whereas the English WP uses citation templates on several millions of pages in the German WP most articles do not use citation templates. --Matthiasb (talk) 10:16, 1 February 2023 (UTC)[reply]
  • Strong oppose MediaWiki should embrace universal worldwide reference standards where they exist, by default, if those standards are clear and explicit, and unambiguous, and international.
It already does this with URLs: a valid URL is interpreted as a link "out of the box" as default behaviour. If I write https://mediawiki.org , it appears as a link by default. If I want mediawiki-specific formatting I can use additional MW link formatting, and if I want totally custom treatment of URLs, I can use a template. If I want completely custom counterintuitive behaviour (like a URL NOT functioning as a link), I can override or use a custom template. This three-or four-tier approach is IMO the correct behaviour – support unambiguous standards by default, support more sophisticated MW behaviour through markup, and support custom behaviour through templates.
IMO, the ISBN identifier is sufficiently explicit that if it appears, MW should support it – Why would a writer include an ISBN number if they didn't want people to be able to look it up?
Also EAN numbers, ORCID numbers, and GPS numbers, doi: numbers, perhaps BCE and CE dates, perhaps universal time code. These standard identifiers can also trigger the auto-insertion of metadata into the html to make pages more easily machine-readable. Yes, we already have Semantic MediaWiki for this, but it's so unwieldy that almost nobody uses it outside of templates.
If anything, we should be taking automatic support for successful standards further. Support for citations is currently a mess (reflecting the existing mess of conflicting citation standards out in the real world), which is why they invented the DOI (Wikipedia:Digital object identifier) standard. We should be embracing doi, using doi-lookup to auto-fill citation fields, auto-finding doi numbers from existing citation templates, and only using the long-winded manual citation fields for doi lookup or for special purposes, such as when no doi is available, when we want more supplementary detail (doi-plus-pagenumber, doi-plus-chaptername), or when the doi database has a problem. ErkDemon (talk) 12:06, 12 April 2024 (UTC)[reply]

See also

[edit]
Retrieved from "https://www.mediawiki.org/w/index.php?title=Requests_for_comment/Future_of_magic_links&oldid=6697752"