MediaWiki Product Insights/Reports/August 2024

Hi All,

Welcome to the monthly MediaWiki Insights email!

First of all, many thanks and congratulations to the 12 new volunteer contributors: Alex (AllUsernamesArePicked), andrybak (Andrei Rybak), AutumnFeather, Claire (BlankEclair), Daniel Riedmueller, Jbol, Mahmoud Al-Qudsi (mqudsi), Newton Kamau, Nunya Klah, Rummskartoffel, XXBlackburnXx, and أنون (Lolekek). Each of them got their first patch merged in MediaWiki core, extensions or services deployed in Wikimedia production during the month of August! (or at the very end of July)

Some of the developers joining us this month are longstanding members of the Wikimedia project communities; others use MediaWiki outside of Wikimedia and submitted patches upstream for the first time; and others we don't know much about, other than that they also wanted to help with the project. Glad to see you all around!

Project Snapshots: Progress on annual plan initiatives

In July we kicked off work under the Foundation’s new annual plan (July 2024 - June 2025).

Two months into the new plan, we’ve made good progress on many of the initiatives to evolve the MediaWiki platform (WE5) and improve the services and tools that developers need to support the Wikimedia projects (WE6).

WE 5.2 is focusing on exploring ways to simplify feature development by enabling more modularity and improving the maintainability of the software. This Key Result ("KR") kicked off with research (first quarter) that will inform the initial experiments (second quarter). We started this work by looking at the ecosystem of hooks that are used by extensions to integrate with MediaWiki. So far, we analyzed a sample of 120 of the 1409 hook handlers we use on MediaWiki sites (implementing 60 of 460 distinct hooks). We evaluated about 20 aspects of each handler in order to identify patterns and opportunities for improvement. For instance, about half of the 60 hooks could be isolated from the database to improve maintainability and security. And it looks like 9 of the hooks could even be replaced by declarative registration, which would reduce complexity. Next, we will summarize our findings and propose experiments and explorations that could lead us towards a more expressive and sustainable interface between MediaWiki core and extensions. You can find more information on the project page.

WE5.3 is focusing on on performance experiments by treating a page as a composition of structured fragments. The initial work consists of two pieces: Instrumentation and data gathering to inform future performance and templating improvements (5.3.1) and Prototype selective html updates in Parsoid (5.3.2).

We have implemented a very basic selective html update technique in Parsoid to update pages when a dependent template is edited. There are still correctness and functionality gaps, but the implementation is complete enough to let us gather preliminary performance numbers. We notice up to 5x+ speedup updating a page when a dependent template is edited - the benefit depends on the complexity of the template and the page using the template. The larger the page and simpler the template, the larger the expected benefit. There is more work to be done to ensure correctness, but we don’t expect this to impact expected performance benefits significantly. This code in Parsoid assumes that the previous version of the HTML is available in cache and the template and page meets the criteria for selective HTML updates.

The selective update technique crucially relies on a set of conditions being satisfied: previous version of page available in parser cache and template generating well-formed DOM fragments. So far, we find that previous HTML is present during parses that account for 50% of time spent in production and we expect to understand how often this is true with the collected data.

Once complete, 5.3.1 and 5.3.2 can let us estimate the benefit of fully implementing and deploying these in production. The early results from 5.3.1 and 5.3.2 look promising.

Separately, we have also been discussing and planning the work required to support the embedding of Wikifunctions on a Parsoid-enabled wiki. This work, when complete, will let Parsoid support Wikifunctions without needing to build a lot of new infrastructure for it.

A previous edition of MediaWiki Insights mentioned work the MediaWiki Interfaces Team has been doing on improving the experience of REST API endpoint creators and callers. Another phase of that work was finished in August with the completion of Rest: Deprecate BodyValidator. This required compatibility changes to several extensions, including: Checklists, MenuEditor, TuleapWikiFarm, ContentStabilization, Workflows, BlueSpiceExtendedSearch and others. Deprecating the old system directs endpoint creators down an implementation path that allows them to take advantage of recent improvements, such as convenience methods to generate response schemas, an initial usage of which can be seen in UpdateHandler. This work is part of KR 5.1: Interventions to increase sustainability of the MediaWiki platform.

A key goal of WE6 (Developer Services) this year is to evolve and improve our software testing capabilities. In the last Insights email we shared about the work that is underway to facilitate the provisioning of testing environments on-demand for automatic and exploratory testing. In the meantime, we have released the new backend for PatchDemo that is backed by Kubernetes. You can try it out by enabling a flag on the PatchDemo UI. Next we will provide full feature completeness to the current offering and enable not just the provisioning of MediaWiki for testing, but also the testing of custom services & extensions. If you don't see your previously created environment, they are still available at patchdemo-legacy.wmcloud.org.

You can read through the full list of initiatives planned under the Knowledge Platform objectives WE5 (MediaWiki Platform) and WE6 (Developer Services) on this page.

With this, we’re wrapping up this edition! It has been great to see some of you at Wikimania in August, at the Future of MediaWiki session, the hallway track and in the dev corner.

Amir Sarabadani just shared with me the link to this ticket, which is an effort primarily driven by Zabe: It started at Wikimania and is about to complete: MCR schema migration for External Store URLs. Many thanks to Zabe & Amir for kicking this off, and for continuing the work after the event!

Thanks all for reading,

--BMueller (WMF) (talk) 16:46, 11 September 2024 (UTC)

P.S.: For those of you who have missed the announcement on other channels: The voting period for the 2024 Board of Trustees elections is still open through September 17, 2024. Eligibility criteria for developers (and for editors and translators) can be found on this page.