Would this replace the existing Developer hub?
Talk:Developer Portal
The Developer hub is about MediaWiki development only, so it might end up as one item being linked from the Wikimedia Developer Portal.
Hi to the dev team,
fyi; when typing the exaxt word (search field at w-w-w-.wiktionary...org)
Akwuise
(the above word was a manually wrong typing of "Akquise" in German)
it causes an error 503. I have tested other words and words nearly by the mistyped word like beginning with small letters etc. and the result was: Other typing does NOT cause internal server error 503.
Kind regards from Germany!
Frank
Hi Frank, this page is about https://developer.wikimedia.org/ only. For issues on Wiktionary please refer to How to report a bug - thanks!
The feature that pops up little previews of links is very useful. But it is not useful when you open such a link in a new TAB in the browser and upon closing said TAB again, that little window is still there and blocks the view to underlying text. You have to find a new link to hover the mouse cursor over to be able to close it.
I usually get so P. O. that I turn that feature off altogether. I guess this was not the intention with its birth. Or perhaps it was? I don't know.
I guess you could say I find it too obtrusive and far too persistent for my taste.
Perhaps a time delay until it is shown, and a timeout when it is automatically closed? If there is, that's far too short in the beginning (showing) and far too long at the end (closing).
Perhaps I'm the only one finding this a very odd behavior. Using Manjaro Linux, Firefox browser. If that matters.
I will probably not be back here, unless I stumble on the topic somehow. Just thought I'd vent my dissatisfaction with this otherwise nice feature.
This is a bit provincial, as I've been working on Parsoid for almost ten years now, but I'd like to see links to Parsoid HTML incorporated somehow. "Working with mediawiki data" is one of the key points here, and it seems like "turning wikitext into a semantically-meaningful HTML document" is a key part of that for half of the folks who want to work with our content. (The other half are probably machine-learning folks who want to strip *all* the markup out and just get raw text out they can feed to a language model, but believe it or not Parsoid HTML can help with that too.) It seems like this should be mentioned in three places:
- There's a link to "MediaWiki and Extensions" in the "PHP language" section, but mediawiki is increasingly composed of numerous independent libraries (like Parsoid) which probably should be called out. Many of these libraries are significantly easier to contribute to than core is.
- The MediaWiki dumps page doesn't mention parsoid-format HTML dumps at all. I know this has a somewhat-twisted history at the org, but my understanding is that the kiwix project is using parsoid HTML from the API, and https://dumps.wikimedia.org/kiwix/zim/wikipedia/ is one way to get at that. There are projects like openzim's mwoffliner which work with this format. T182351 is a newish bug (2017!) for this, T17017 (2008) is the earliest I could find in a short search; T302237 is a task from this year which uses the Wikimedia Enterprise HTML dumps which are indeed in parsoid HTML.
- There's a detailed specification for the HTML generated by MediaWiki at Specs/HTML/. I don't know exactly where that should live in the developer site, but it seems like "working with HTML" belongs somewhere. That's what gadget authors will see, and it will be increasingly visible to readers as Parsoid read views rolls out.
I'm sure some of the responsibility here falls on the content transform team for not making 'excellent' user-friendly documentation for some of these tools/formats. So perhaps the question is: how do we fix this? What's the bar for inclusion, and how do we get creating the necessary documentation resourced?
Hello. I took a look at <https://developer.wikimedia.org/> and I'm very confused what took two years and why having a separate static site is beneficial at all.
We already have a "Developers" link in the footer of every Wikimedia wiki that points to How to contribute. This wiki page is fully translated and even includes cute black and white icons for each subsection.
This static site is a major step backward. It's a bunch of additional complexity in order to........... output some headings, description text, and hyperlinks? That's exactly what MediaWiki does best and is already doing for this exact purpose.
Why have a static site when the entire point of using a wiki is to have a dynamic and collaborative site? Instead of being able to edit a wiki page, which we actively want people to do, you now need to file a Phabricator task to make content changes?! This is beyond the pale for a wiki organization.
I don't understand why this project was undertaken or why so many people were involved in remaking something that already exists, but worse.
The silence is deafening, as they say.
Of course it doesn’t take two years to build a static site generator :-).
Developer Portal is a tool to help show and build paths through Wikimedia’s technical documentation, which is distributed across wikis, code repositories and other sites. It’s part of a broader initiative to systematically improve documentation (see also: Dev Portal launch email).
Beyond the tasks directly related to the Dev Portal implementation - research, design and technical implementation - this includes creating review guidelines, conducting content reviews, talking with subject matter experts (people who are key contributors to a page/experts of a topic), updating and improving docs, consolidating pages or thinking about information architecture and location of docs (is it helpful to have 5 different pages on topic X, each of them with slightly differing information, or should there be one page? Is mediawiki.org the best place for documenting topic Y, or should that live elsewhere? - etc.). More info can be found on the overview and sub pages, blog posts (techblog, diff) and the phab boards for the Dev Portal and key docs updates.
On using a static site generator vs using an existing or new wiki: MediaWiki is an extremely powerful software and great for collecting and structuring content. However, the Dev Portal is a navigation tool that covers multiple curated user journeys. It doesn’t contain the actual technical documentation, but links to technical information that is distributed across wikis, code repositories, other sites and more. Content on the Dev Portal therefore is much less frequently subject to additions/removals than wiki content is, and changes should be done with the design principles and user journeys in mind. The process to update a curated journey structure is much closer to the code contribution and review process than to the editing process. Having this link hub on mediawiki.org or wikitech wiki would be like building the front door to the house into the middle of the living room or kitchen. If Wikimedia technologies and ways to contribute for developers would only compass MediaWiki you’d be right - the door to entry should be on mediawiki.org in that case.
Hmm. So the goal of the separate static site is to introduce a gatekeeper into the process, forcing a Cathedral-style change review for the navigation?
Is the lack of such a gatekeeper really an identified weakness in the current process? And if it is, is its impact really of such a magnitude that it justifies the resource spend on developing Yet Another CMS™ to manage a single aspect of the process? Is such a process sustainable over time given the vast scope of documentation and audiences? Is it sufficiently accessible for all stakeholders, given a huge number of them are primarily on-wiki contributors who do not have experience with code-review, are not familiar with Phabricator, and lack the skills to "set up your #Local development environment and provide a changeset in a Gerrit branch following the commit message guidelines."?
I am failing to see the technical or process factors why all the documented functionality could not be implemented on-wiki in MediaWiki proper. And that would have the significant benefit of lowering the barrier to contribute, enabling drawing on crowd-sourced expertise and capacity, and improved dog-fooding.
Alternately, if this projects goals are primarily easier onboarding of new WMF-employed developers or consultants its project documentation should probably be updated to reflect this more clearly to avoid wasting volunteer effort on it.
@Xover The current setup is not a CMS™. It is a static site that offers a quite limited collection of links and short link descriptions, categorized based on research on user journeys. The content itself is rather static and simple compared to wikis. I would not set up and maintain a huge toolkit (MediaWiki) if I only needed a hammer to put a nail into the wall (after thoroughly checking the wall).
The target audience of the Developer Portal are existing and future developers who want to technically contribute (or want to link to their shared technical knowledge), not any general "on-wiki contributors". Thus to contribute to the Developer Portal itself I'd expect you to already be a bit familiar with developer workflows such as Phabricator or Gerrit.
I'd love to see "crowd-source expertise and capacity" when it comes to editing the resources linked from the Developer Portal. Because there is always on-wiki documentation which is in a neglected state. That's where help is welcome and needed.
As an unrelated example, I've spent the last days trying to make the MediaWiki installation guides' (plural!) pages on mediawiki.org not quadruplicate [sic!] stuff but allow people to actually understand what they need to do without getting lost on a dozen [sic!] of pages with partially outdated and confusing information. I assume that there has not been any "gatekeeper" around for many years. Wisdom of the crowd is a lovely narrative. A crowd though requires a number of people who have a good understanding, an overview, and [time to] care.
@AKlapper (WMF): But what you're describing there is not a gatekeeper, but rather leadership, coordination, and design. My point is that that's exactly what is most desperately needed, and I don't understand what benefit this gatekeeper mechanism will bring us or what actual problem it will solve.
And I don't understand why we need a separate static site for "a quite limited collection of links and short link descriptions, categorized based on research on user journeys". What part of that can not be implemented on one of our existing wikis (e.g. Wikitech) using our biggest and most core product (i.e. MediaWiki)? It's kinda exactly what wikis are designed to do, and if it can't, using the resources on dogfooding this would benefit the core product.
And I don't understand why I need to already be a developer—here apparently defined as "MediaWiki developer, with a full Docker MW pipeline set up, and at least +1 on core"—to be able to fix a typo in one of those "short descriptions", give feedback on a "user journey" that's missing, or suggest a new link.
I have code deployed on several hundred WMF wikis (and in pywikibot, and, depending on what distro you're running, in coreutils and possibly even the kernel on WMF servers), spend the majority of my volunteer time writing code, doing technical writing, or trying not to annoy the bug wrangler too badly while looking for the right team tag to take a given Phab task (Multimedia, anyone? Anyone? Hello...?). But I am never going to have +2 on core, simply because I can't afford the time investment in being able to shepherd something through Gerrit, the CI, the reviews, through SRE and the deploy train, what happens if something breaks, etc. etc. That is, I am a "developer" in the sense that I can and do write code and technical documentation, but I am also in general an "on-wiki contributor".
That means I am not eligible for Community representation in the Technical decision making forum. I am not, apparently, one of the target audiences for knowing what the path forward for OOUI-based Gadgets is (and, see previous point, am not a stakeholder in Codex or Vue; more fool me for having invested time into OOUI-based Gadgets). And I apparently have nothing conceivably to contribute to the Developer Portal...
If someone wants to tell me straight up that I am not the intended audience for this effort and I have nothing worthwhile to contribute then I can accept that (I'll disagree, strongly, but fair enough), but until then I am failing to understand why deliberately raising the barrier for participation and inventing a better wheel to enforce that is either necessary or desirable. In other words, I'm asking "Who's it for, and what problem is it solving?".
@Xover You could implement what you describe in one of our existing wikis, however it is not a good idea. It will create the problem that BMueller described in the comment above: building the front door to the house into the middle of the living room.
I don't see what the Technical Decision Making forum (I have not been following much on that, I admit) has to do with the Developer Portal, or where your definition of "MediaWiki developer, with a full Docker MW pipeline set up, and at least +1 on core" comes from when it comes to the Developer Portal?
You are a developer, a very active technical contributor, and you are clearly part of the intended audience of the Developer Portal ("anyone interacting with Wikimedia code"). Your input on user journeys or potential links that are missing is welcome.
@AKlapper (WMF): I'm not sure these ad hoc analogies are useful; or at least I don't understand what is supposed to be the "door" and the "house" in this one. To navigate a building the signage needs to be somewhere visible when you approach or enter the house, and it helps you navigate to your desired destination. Having to go down the street to look at a stand-alone sign seems less useful. Having dynamic signage that can update immediately when stuff moves around in the building seems useful too, instead of a static map you can go look at at the tourist office.
Meanwhile, WMF wikis have numerous navigation features, including portals, start and landing pages, navboxes, categories, outlines, overview articles, etc. On English Wikipedia they help you efficiently navigate somewhere north of 5 million articles, all of them no more than six clicks away from Philosophy. Content on one page can be reused on another page through transclusion or Labeled Section Transclusion. All changes are version-controlled and can be easily reverted. You can even have a gatekeeper if absolutely needed through pending revisions or page protection or semi-protection, or a special user group like template editors, file movers, or interface editors.
Which is why I don't understand what the problem with implementing this on a wiki and as a wiki is, or what benefit doing it outside as a static site brings.
I understand, conceptually, the desire to design a process with a gatekeeper (which is an orthogonal issue to technical implementation), but I don't understand why it is felt that it is needed, what actual problem it solves, or what benefit it brings. It prevents more people from writing documentation? It keeps that pesky "everyone" from editing the site "…that anyone can edit"?
"to contribute to the Developer Portal itself I'd expect you to already be a bit familiar with developer workflows such as Phabricator or Gerrit." To fix a typo?!? That's not your average on-wiki contributor. I would be able to navigate both, but am I really going to bother? The tech decision forum has no community representation, but they have just opened up the possibility: to community members with +2 on the MW ACL. How many primarily on-wiki contributors have +2? Even submitting a simple typo fix as a patch in Gerrit requires trawling through tons of documentation (I speak from experience), unique to Mediawiki, and unless you do it regularly you end up starting from scratch each time.
Meanwhile, I just followed the link you posted in T279421 and noticed it had a missing closing parenthesis, so I just fixed it. It took all of ten seconds so it wasn't no trouble...
@Xover Have you visited the Developer Portal? I'm asking as I can't follow "preventing more people from writing documentation". There is intentionally no documentation on the Developer Portal itself but only links and short descriptions. How could it prevent people from writing documentation then? To the contrary, it points to those pages that are the most important technical documentation resources - please help edit and improve them! :)
The content on the Developer Portal itself is extremely static, so we expect only a small number of changes over time. The Developer Portal does not need navboxes, categories, overview articles, transclusions, pending revisions, etc. It's intentionally a clean and limited experience with less distraction (and a clear path forward to ideally 1 resource that covers a topic, avoiding anti-patterns like e.g. lots and lots of links on a wiki page that make folks easily end up down in a rabbit hole).
(PS: As I wrote before I do not know why the Tech Decision Forum is mentioned again. It has nothing to do with the Developer Portal at all.)
@AKlapper (WMF): We seem to be failing to communicate effectively here.
I am asking for the specific reasons why we had to spend money and scarce human resources on developing a separate and custom solution that can't reuse existing volunteer (and staff, for that matter) skill sets, instead of just implementing the relevant content on an existing wiki using our core technical component (MediaWiki).
I am also asking what problem the increased barrier to contributing and the gatekeeping solves over simply letting anyone edit it directly (like on, you know, Wikipedia). What is the benefit derived by deviating from this core tenet of the Movement?
So far I am failing to find answers to these questions.
I have indeed visited the developer portal. Every page of it. I see nothing there that couldn't equally well be on a wiki page. What am I missing?
Because it "just" would not solve the problem. I'm afraid I don't know how else to express it than how I've already tried here, sorry. :-(
I'm just wondering if there's any ordering to the languages listed at https://developer.wikimedia.org/contribute/by-language/ — I'm totally biased, but I'd have thought PHP would be higher up the list! :-)
Also, should Lua be on the list too?
I find the order sensible. I think it's written from the perspective of an interested volunteer developer: Where should I start when I never touched any MediaWiki-related code? I would probably not suggest core but something like Pywikibot.
We based the ordering on user research interviews that we did. Although PHP is an important part of the Wikimedia ecosystem, it ranked lower than other programming languages with the people we talked to.
We would like to include Lua but didn't find any Lua projects with clear contributing instructions. There's a high barrier to entry for writing Lua for MediaWiki via Scribunto, so we opted not to include that in favor of projects with a more traditional open-source contributing process. Thanks for the questions!
One thing I think that can be highlighted better are the replication dbs accessible from tools. I often see that ppl don't really know why they should be using toolforge and this is one of the primary reasons to use toolforge to do something you otherwise likely wouldn't be able to. I see talk about tools and about datadumps, even quarry, but not the rep dbs which are kinda in between and a bit lost in this picture I feel.
++ aka "how is global search so much faster than single-site search?" :)
Great point! Do you think wikitech:Help:Toolforge/Database would be the best resource to highlight?
namaste to wikipedi context department :
can you change the your prefer context into my indian i neddd transcript languege where as hindhi telugu marathi gujrati and other language need to under stand easiy will you can for it please
Sorry, I don't understand what is requested where exactly. Please elaborate, and provide exact steps to reproduce, and full links to the page that you refer to. Thanks!
This looks great. Thank you for the update // the code + text search tools are so useful, any updates to those would have huge impact especially now that they are so prominently displayed.
I'm excited to have code search and cross-site search more visible as well! I find them very useful. To make sure I understand your comment, you're referring to updates and improvements to those sites? Not to the way they're displayed in the Dev Portal?
Yes, I'm suggesting that these are high-leverage tools :) relatively simple improvements to search experience could greatly improve life + speed work + avoid duplication for many people (once everyone knows they exist). For instance: could reduce orphaned projects and lead more people to adopt + update existing work before creating new things
I see the scope doesn't include writing or re-organizing new documentation. Maybe there could be some parallel initiative, though, to include the community to do this work?
For example, if a developer wants to learn how to write PHPUnit tests, what would the developer portal link them to? Manual:PHP unit testing already has a bunch of links, where the linked pages overlap in what they discuss, a lot of the examples are out of date, etc.
Anyone is free and welcome to start or support parallel initiatives - Wikimedia technical_documentation:_Friends_of_the_Docs might already be a good (and existing) place for anybody interested in improving Wikimedia documentation.
Which exact content to list in a Wikimedia Developer Portal which and which pages to link from a Wikimedia Developer Portal are one of the many things to sort out as we progress. Providing info about PHPUnit tests might be a listed item.
@KHarlan (WMF) I probably came across as way too "out of scope" in my previous comment, and need to correct myself. As part of the process we will have to make sure that the "key technical documents" linked from a future developer portal are well organized and up-to-date, and that it's clear who is responsible for each document. I have created phab:T264080 about that, to be tackled somewhere around the middle of the 2021 calendar year. Thanks again for the comment!
If anyone is interested, I'm currently working on a new guide for writing extensions. My progress for both pages (although still both WIPs) are at User:SamanthaNguyen/Guides/Writing an extension and User:SamanthaNguyen/Guides/Writing special pages. I'm currently not including parser tags/extension tags since that is now being covered by Parsoid (and I believe magic words also fall under this). Once those are closer to finished, I'll start re-expanding the pages related to content models and content handlers. Both pages assume nothing, so as to make sure it's helpful for everyone. Its meant so that those who are more experienced can simply skip sections as need be.
@SamanthaNguyen Those two links look like quite good tutorials to me (speaking as a non-developer, I understand what I'm supposed to do way better than on many of the existing pages). Thanks for working on this! I would be glad to see them moved out of the user namespace at some point! :)