Topic on Talk:Stable interface policy/Archive 2

The policy currently states:

• Deprecations and removals SHOULD NOT be performed shortly before a release branch or between release candidates, to give extension authors time to fix any issues that may arise, and avoid broken snapshots of extensions.
• Deprecations and removals MUST first take place on the master branch. Deprecations and removals SHOULD NOT be backported to release branches or release candidates.

It seems like the first bullet point is redundant and misleading: we can still do deprecations on the master branch while release candidates are being prepared. We just shouldn't add deprecations to the branch that is going to be released. Since that is already covered by the wording in the second point, I think the first point should simply be removed.

Agreed on the point about "being prepared", but the main part should stay. It's really unhelpful to make large breaking changes (which deprecations are, in effect) a few days or hours before a branch is cut, and doing so just afterwards can make backports very hard.

Should we be more specific about what "shortly" means? A week should do, right? We'd need some kind of official "pencils down" warning for this to work.

No, not just a week. I'd recommend a month before and after the branch point, generally. This isn't about Wikimedia staff fixing things (where a week's notice is very tight but might be OK), this is about the wider ecosystem, right? People might not even notice that their extension no longer passes CI for a week or two after the deprecation, let alone fix it.

That would mean not making deprecations in two out of six months. That seems like a massive impediment. Can't we just run CI for all of them?

No? Running >>US$1k worth of CI CPU time for every patch to MediaWiki core, and then doing something with that result, would be a massive change to our CI budget and expectations for developers. (If a patch in core causes a failure in an extension, how is that communicated? To whom? How do maintainers find out that they need to do something? How do core developers not end up deluged by false positive "failure" messages from the dozens of extensions that are broken already? Etc.)

Also, this is just documenting current expectations, it's not a change to the current reality, AIUI?

Not doing deprecations for a month before and after a branch cut is definitely not current reality, and never has been as far as I can remember.

My suggestion was to run CI on all of them when doing the rc0 branch, not on every patch. My hope is that this would result in extension maintainers noticing and fixing issues before the actual release.

I mean, I often C-1 patches that would be disruptive, as do many of the usual crowd. I'd encourage you to consider doing the same, out of respect for the time of colleagues and particularly the Security team when making last-minute back-ports.

The wider suggestion is an interesting idea. You should pitch it to RelEng, though I imagine they'd want to defer such work until after the GitLab migration rather than implementing such a tool from scratch twice within a year.

To clarify: I agree that we should not do backports shortly before a release. Deprecations and removals should not be backported at all. But doing a deprecation on master a week before or after the rc0 branch is cut seems fine to me. I'm not sure I understand the problem with that. Extensions authors still have a month to fix problems between rc0 and the actual release, right? Maybe I'm misunderstanding when and how extensions snapshots are made...

As to running CI for all extensions... it wouldn't be hard to write a script that pushes a DNM patch to every repo, right? And maybe even automatically file tickets if these patches fail to pass CI...

To clarify: I agree that we should not do backports shortly before a release. Deprecations and removals should not be backported at all. But doing a deprecation on master a week before or after the rc0 branch is cut seems fine to me. I'm not sure I understand the problem with that. Extensions authors still have a month to fix problems between rc0 and the actual release, right? Maybe I'm misunderstanding when and how extensions snapshots are made...

There are two sources of pain:

• People making big changes "just" after the branch is cut, before it's released. This pain mostly affects MediaWiki itself and deployed extensions.
• People making big changes "just" before the branch is cut. This pain mostly affects those extension developers who aren't pushing a patch every few hours.

Branches are cut from master at the same time or very shortly after the "final" alpha deployment branch is, for all ~1800 extensions and skins (not just the ~200 in Wikimedia production). After that, there's a period of manual testing for things that no-one cares about except in releases, like the installer, and a bunch of bugs are found and fixed. As with all Wikimedia development, those changes are made first to the master branch and then back-ported. We also backport major bug fixes that are discovered during the week of the deployment. The general guidance is that this post-cut process can take up to six weeks until rc.0 is cut, though in practice I'd not worry about it much beyond the first three–four. Naturally, LTS releases make for even more pressure to get things out of the door, and so rc.0 can unveil lots of bugs and need several follow-ups (as we found with the 1.35 release), but in general it's pretty free sailing after rc.0 is released.

For the first area of pain, large scale refactors or removals of code very frequently mean that these back-ports can't be easily made, or in some cases have to be re-written from scratch. In general the burden of this churn doesn't fall on the person that made the work by merging the refactor so swiftly, but on the bug fixer (or worse, someone like me just trying to help the release get out).

For the second area of pain, hard deprecations are, per long-standing development policy, effectively a breaking change for development (and for Wikimedia-deployed code), as they make CI fail. If one were e.g. to hard-deprecate the User class a week before REL1_37 were branched, hundreds of extensions would immediately be broken for development, and either their owners would find out a month or two later when they next pushed a patch, or (more frequently) helpers like me or tools like LibraryUpgrader would encounter it when fixing up the repo for other reasons.

I think this section should be split into statements about breaking changes and deprecations, and say different things about them, e.g.:

• Breaking changes and large or risky refactors SHOULD NOT be performed during the few weeks running up to a release being branched until the first release candidate is made, to make it easier to back-port bug fixes to release candidates.
• Hard deprecations SHOULD NOT be performed shortly before a release branch, to give extension authors time to fix any issues that may arise.

• Deprecations and removals MUST first take place on the master branch. Deprecations and removals SHOULD NOT be backported to release branches after the first release candidate is made.

Would that work?

Maybe I'm thinking about this wrong, but in my mind, the best time for extension developers to discover and fix issues is after the branch has been cut, so it's no longer a moving target. I assumed ExtensionDistributor will pick up any fixed made to the extension's snapshot branch. In my understanding, this would make sure that at the time of the release, the snapshot branches of extensions have also been fixed. To support this, CI should run against the extension when the branch is created.

As to not making major changes between the branching and the release of rc0: I think this is reasonable, but would need to be announced more clearly. Also, having at least a rough idea of the timing beforehand would allow teams to plan around this.