Development policy/Until 2018

The MediaWiki software is developed collaboratively by hackers from all around the world. To make sure that none of WMF's hundreds of wikis breaks by an upgrade, but to also make sure that all our wikis are reasonably up to date, we have a general development policy, outlined below.

Gerrit access

Development is done using Gerrit to manage our git repositories. To gain access to Gerrit, our code review system, anyone can register at wikitech:Special:UserLogin/signup. Git with Gerrit is a lot like wiki with FlaggedRevs: Everything must be reviewed and approved, and can be reverted.

Master branch as unstable and Wikimedia deployments

Main development takes place in the master branch of Git; it is unstable, but must always be kept in a working state. See also +2. Major modifications should take place in a development branch and only be merged when ready.

Code is automatically synced from the git master branch to a staging cluster ("Beta Cluster"), generally within a few minutes of being merged. This allows further testing before deployment as an additional layer of defence against bugs; it is not a substitute for testing in advance of merge.

Code is generally deployed to production in stages. The WMF git branch is made from master, and this is released to the Test Wikipedia, the Test2 Wikipedia, the Test Wikidata, and MediaWiki.org before it goes live on other production sites. This is to ensure that everything isn't taken out by the update. It is important to remember that this is not a substitute for testing and all code should thoroughly be tested before being merged into Git master.

Releases

Every few months, the master branch is cut into a release branch of the form REL*. If you care about release management or want to try out one of the stable versions, these are the only branches that matter. Typically there will be a testing phase when a new release is begun. There might be a few release candidates, a 1.x.0 version, and finally a few minor bugfix releases. Releases are also packaged into .tar.gz files and made accessible from the releases.wikimedia.org webserver.

This ensures that users of our software can easily get a reasonably stable version without messing with Git. Anyone submitting patches should preferably use the latest version of master to avoid conflicts, and of course people with commit access need to use Gerrit to be able to push stuff for review. It's inadvisable to run production wikis off of development versions unless you know what you're doing (as do, for instance, the Wikimedia tech staff, a number of whom are also MediaWiki developers).

The Release checklist provides a more comprehensive list of tasks to be done for a release.

Database policy

Per the RFC "Increase the strictness of MediaWiki SQL code" (phab:T112637):

WMF will be enabling MariaDB/MySQL's strict mode (phab:T108255), which will be the default anyway in MySQL 5.7. Prior to this, all new code must not generate any warning.
Code that touches the database must be compatible with the following MySQL SQL modes:
- TRADITIONAL (equivalent to STRICT_TRANS_TABLES, STRICT_ALL_TABLES, NO_ZERO_IN_DATE, NO_ZERO_DATE, ERROR_FOR_DIVISION_BY_ZERO, NO_AUTO_CREATE_USER)
- ONLY_FULL_GROUP_BY
Database code must be compatible with older versions of databases as listed in the MediaWiki installation requirements for database server (e.g. the stable MediaWiki release requires MySQL 5.7.0 or later). However, performance improvements that would only apply to the newest, supported versions (or its default or widely recommended defaults) should be favoured over those that apply only to unsupported releases (5.1 and lower).
All tables must have a primary key. When a candidate for primary key could not be created (for example, if all columns can be repeated), an auto_increment or another arbitrary value, depending on the case, has to be added.
Non-deterministic queries and unsafe statements for binlog should be avoided as they would return/write different results in a replication environment. The latter can be detected as warnings with the text "[Warning] Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT". Those include INSERT ... SELECT when using an auto_increment key, UPDATE ... LIMIT without an ORDER BY, and using non-deterministic functions like SYSDATE(). https://dev.mysql.com/doc/refman/5.6/en/replication-rbr-safe-unsafe.html has more info
Primary keys, and fields that reference them, should be unsigned, in order to increase the maximum values. For more details, see http://www.percona.com/files/presentations/WEBINAR-MySQL-Indexing-Best-Practices.pdf . Database optimization practices should also be followed.

Database patches

If you change the database schema, follow these rules:

Update the installer – Please update includes/installer and add an appropriate SQL update file to maintenance/archives/. The naming convention, if you're adding a field, is patch-table-field.sql. If you're removing a field, it's patch-drop-table-field.sql. If you're adding a table, it's patch-table.sql. Look at the commit history of includes/installer/(MySQL|Postgres|SQLite|Mssql|Oracle)Updater.php to find examples of how it's done. If you're adding a bunch of fields to the same table, make all those changes in one query in one patch file.
Make your schema change optional – All schema changes must go through a period of being optional. Some examples:
- Instead of changing the format of a column, create a new column, make all writes happen to the old and new column (if it exists) and deprecate use of the old column. Check if the new column exists before blindly assuming that it does. Only eliminate support for the old column after it's clear the schema migration has completed and there's no chance that we'll need to roll back to the old version of the software. If this doesn't seem feasible, send mail to wikitech-l asking for advice.
- You could set your new feature to only work if a config option is set to true, and set the option to false by default. Then the commit can be safely deployed before the schema change is made. To deploy your feature to the Wikimedia cluster, file a ticket in Phabricator in the relevant project with the #schema-change tag. Once you've confirmed the change has been made, you can remove the config option to enable your feature.
- Note that this means your schema change should be optional in code - for wikimedia deployments, it is expected that every wiki with the relevant database table(s) will have the schema change applied to them. If you need different schema for different wikis, then apply the change using an extension and creating new tables dependent on that extension.

There might be cases where the "make your schema change optional" rule will be prohibitive from a performance or logistics perspective. However, schema changes like that should be rare to begin with, and should have prominent discussion on the wikitech-l mailing list. In the case where it's impossible to make your schema change optional, it is still critical to write scripts to roll back to the pre-change state.

Search for input from a WMF Database Administrator – MediaWiki is deployed to Wikimedia websites every week, and it takes considerable planning to apply schema changes to MySQL-based sites the size of Wikipedia. As of September 2017, Jaime Crespo (jcrespo on LDAP, jynus on irc and Manuel Arostegui, marostegui) are the best people to add to database reviews. In most cases, input is just needed on the logistics of the change.

Test your changes on Beta - in particular, it is a common mistake to change indexes and column definitions that would result in different query plans. Try to test the generated queries' plan with tools such as EXPLAIN; not doing so could mean that, when scaled to production, queries that only take 1 second locally, they pileup on production when they receive much more traffic and have larger tables.

Deploying to production

Once a migration has been reviewed and agreed, if it requires WMF deployment, please follow the please Deploying schema changes policy on wikitech; which basically means creating a new Phabricator task with the #Blocked-on-schema-change tag with all necessary details, then get in touch for deployment scheduling. A one week of leading time is asked, more in case it affects one of the core tables (page, revision, image on Commons) or one of the most used/largest ones (*links tables). Ask for an estimation if you are in doubt.

Test wiki policy

There is an official test wiki at https://test.wikipedia.org. Some developers may also have test servers deployed.

Generally, the purpose of such "fresh from Git" servers is to test the latest unstable branch of the code, not to have a testbed for demonstrating unfinished features. Features should be demonstrated with GUI mock-ups before they are implemented (these can be uploaded to Wikimedia Commons), and implementations should only be pushed to Gerrit if they are in a reasonably complete state. This ensures that we avoid features that are never finished. To test your own code, please make sure you have a fully working local MediaWiki installation; if you want others to test your unstable code, you can always set up your own test server.

Documentation policy

If a feature is going to be visible to end users or administrators, it's important that the developer documents the feature. Broadly speaking, you can document

in code comments as part of your code change that turn into generated PHP and JavaScript documentation at https://doc.wikimedia.org
in updates to the RELEASE NOTES file as part of your code change to MediaWiki core
in text files in git as part of your code change
in the vast trove of general MediaWiki documentation on this wiki
in wiki pages on Wikitech that describe how MediaWiki is set up on the Wikimedia sites.

Just a few sentences are fine: what it does, how to use it. Others will fix your prose up – the important part is to have features documented. Documentation/Style guide has some suggestions.

Wiki pages on www.mediawiki.org use templates so that you just have to edit a few lines and the template will handle everything else. As the developer, you are the subject matter expert and therefore possibly in a better position than anyone else to document the feature accurately and thoroughly, especially if some aspects of it are not particularly self-explanatory. If you're too lazy, on the other hand, there is fortunately at least the theoretical possibility that someone will be willing and able to do it for you, since this is an open-edit wiki.