Talk:The Tech Debt Trap

I think talking about "technical debt" is misleading: it makes people think about tech debt they are creating, which is not really the root of the problem.

The thing is, code requires constant maintenance. Standards evolve, underlying libraries and languages change, expectations shift; what was considered cutting edge 10 years ago is confusing and archaic today. Keeping a codebase modern is a constant effort, even if it originally wasn't written in a tech-debt-inducing way. So it's less like debt and more like infrastructure maintenance (think roads, buildings etc): you can either continuously make small upgrades, or wait until the whole thing has decayed to the point of uselessness, at which point fixing will be much more expensive (maybe prohibitively so).

That's the central issue with MediaWiki IMO, that those constant updates aren't really happening. RfCs are not consistently implemented, there are shifts in how we think about the architecture but they never get fully implemented (like the API/DerivativeRequest issue where everyone agrees how our API code should look and then things are left at that). There is no social contract on who should do that kind of maintenance: the people who use the code to build features with it, or some dedicated group (that currently doesn't really exist). Tech debt (in the sense of people cutting corners intentionally) is sometimes a problem, but it's not the main problem.

You are right, originally, technical debt referred to debt taken on intentionally. It seems the meaning has shifted to refer to any kind of structural problem with legacy code, anything that slows down development and makes maintenance expensive.

I like Ward Cunningham's proposal to use "acceptable debt" for the kind of debt taken on consciously. What term should we use in contrast, to mean the kind that has accumulated over time? Rot? Liability? Cruft of coders past?...

This sounds great but how do we start to improve the situation? Maybe start with filling in Developers/Maintainers so there are no red squares (there are a lot of them now)?

There's also https://phabricator.wikimedia.org/project/view/609/ - should we track it? Should we plan for some kind of regular progress on it, generally or per team?

Per my reply under the Unclear Ownership thread, there's some work that started recently to try and address this. Task https://phabricator.wikimedia.org/T174091is specific to the stewardship work.

A concerted effort may be good, but it may also be overwhelming.

My suggestion would be to at least assign ownership and schedule tech debt wrangling when the need arises. If the editors team runs into an issue with the editpage API, they should take ownership of that API at that point, and find out what it takes to fix it. If it turns out that this needs a change to, say, the revision storage layer, they should talk to the platform team.

Picking one refactoring that we have already agreed on but haven't properly finished (say, extension registration) and actually driving it to the end could be a possible avenue for more organized activity. Right now architectural improvements usually get stuck in "we now have N+1 competing standards" land.

Other topic pages of interest may also be:

Code Health

Technical Debt Program

Technical debt

``` But I believe the underlying fundamental issue is unclear ownership. Programmers constantly work with layers of abstraction (interfaces): they build their own layer on top of an existing layer, with the top layer being the user interface. But often, they underlying layer, the foundation that we are trying to build our feature on, has problems. It may offer insufficient flexibility, or insufficient guarantees, or may simply be very hard to use. So, who's problem is that? Who needs to fix it? ```

Isn't that why we set up a MediaWiki Platform team?

A feature team / department and platform team / department of roughly equal size would be a reasonable foundation for that scenario. In reality, Audiences is something like 50 engineers, and the Platform team is 3 and a half.

Yes, what Tgr said - this is not remotely possible with the current staffing. Also, if the "backend" is use-case specific (as is the case for e.g. for the watchlist), it makes sense for the "feature" team to take ownership. Otherwise, we get feature teams adding tech debt "for now", and some poor cleanup team trying to fix it...

In a true DevOps shop, the feature team would not opnly own the api code, it would even own and run the corresponding (micro)service. That's why "full stack" is all the rage now, I guess...

I personally think true full stack developers are so rare, they are effectively a myth - but full stack teams make a lot of sense. That doesn't mean I think everybody should do their own plumbing, of course - horizontals (cross-cutting concerns) should have clear ownership as well. Communication and knowledge transfer between teams is key.

Yeah, that was the idea (as advertised, at least) with the big reorg two years ago: vertical auidence-specific departments which take responsibility for the full architecture stack they use, from client-side code to operations. Except in actual practice core maintenance got dropped on the floor so hard we are still gathering the pieces.

One of the things that we are working on as part of the Tech Debt program is identifying "stewards" with the goal of identifying ownership. That being said, one of the first questions I get with asking if they are "stewards" is what it means to be a steward. To that end I've started working on that a bit, see: Development policy/Code Stewardship. I'd be curious to see if you've got some thoughts about what it means to be a steward.

I also made some edits (later reverted), to the developers/maintainers to start pulling in the concept of stewards.

For what it's worth, I believe that lack of ownership/stewardship is a challenge that contributes to a number of things and should be addressed.