Bug management/Development prioritization

This is a high-level essay about software development. It is meant to help people better understand how planning works. (It does not and cannot list every single software maintainer's or engineering team's workflow details though.)

Basics

In free and open source software projects like Wikimedia's software projects, all software code is public. A public task tracking system (like Phabricator) is used to manage software bug reports and feature requests. Anyone can report bugs or request features or propose code contributions.

Why has nobody fixed this issue yet?

Individuals, third-party stakeholders (who use Wikimedia software projects for their needs), and members of several organizations (Wikimedia Foundation, chapters like Wikimedia Deutschland or Wikimedia Sverige, etc.) work on the code of software projects. All of them have their own interests, priorities, goals, and motivations. Some developers might write code to "scratch their personal itch", other developers might write code because their organization wants to see a certain functionality in the software.

Many individuals and groups with different interests exist when it comes to activities (readers, editors, translators, documentation writers, developers, media uploaders, designers, etc.), roles (stewards, oversighters, admins, etc.), preferred human languages and how well these languages are supported in the software, Wikimedia projects interested in, the type of implementation of the software (gadgets and user scripts, modules, templates, code repositories, bots, tools on Toolforge), etc. It is not easy to cover and balance all those different needs of different people and groups in ways which always feel best to everybody.

In theory and in the long run, people and time are unlimited, as more people could start contributing to a project at any time. In practice and in the short run, both people and time are limited resources. Teams organize and plan their work in certain ways, for example as sprints. Adding unplanned additional work to a sprint usually only happens when urgent problems appear.

Furthermore, developers and maintainers may lose interest in a project or have less time due to exams, job changes, children, or other personal reasons. Maintainers may not always realize and thus don't pass maintainership gracefully by finding a successor. Or organizations may change plans and reassign developers to different projects.

As the task tracking system is public and anyone can propose anything, the number of ideas and feature requests is usually higher than the number of developers working on them. That means that the number of open tasks grows over the years.

Some tasks might also be very complicated to fix (software architecture issues where efforts might outweigh gain), considered too risky, or the maintainers consider it out of scope of the functionality that the software should cover.

Example from phab:T208494: Adding support for another file format for music scores would require security screening, its own validation, its own rendering, its own playback, its own set of tests, its own interface messages, its own server-side dependency management, and so on. Fixing security issues, other bugs, testing, and hardware growth requirements add maintenance costs to cover in the future.

Even if someone wrote and proposed code which implements a feature, there might not be resources to maintain the costs of the additional code in the long run. Adding code adds more complexity: someone needs to update it when other code that it relies on has changed. Some bugs only happen when a certain combination of options is enabled which makes it harder for developers to reproduce these bugs.

Hence there are many tasks which will stay unresolved forever. It is unrealistic to expect all tasks to ever get fixed.

In Wikimedia Phabricator, anyone could set the priority of a task. This can create wrong expectations, as priorities should only be set by those who do the work or own a position to plan work and not by someone requesting work to be done by someone else. Furthermore, not every developer or team necessarily uses the Priority field for their planning – some might prefer using the workboard columns in Phabricator or other ways to plan what's being worked on next. Hence it is not a problem per se that many tasks do not have a Priority set.

If you really want something to be implemented, sometimes the best bet is to provide a software patch and getting it reviewed or to convince someone else (maintainers, developers, external developers) to write that software patch. (Making code review happen can be a follow-up problem though. Code review of proposed patches is out of scope for this essay.)

Why wasn't I consulted about these changes?

With half a billion users, it is not reasonable to expect developers to ask everyone about every change (or even trying to reach consensus). It would heavily slow down any development. The problems of the silent majority and self-selection bias exist, and if you try to apply the most popular opinion by pleasing the most vocal people, you often end up with a wrong answer. Furthermore, future Wikimedia community members cannot express their needs because these people are simply not yet part of our community.

There are several ways to learn about new developments. A lot of activity happens and is documented in public:

• For proposed changes with a major impact on people's workflows, research is usually performed beforehand (for example by running A/B tests).
• On-wiki consultations take place to make decisions about major software features or new products, often supported by the Movement Communications team of the Wikimedia Foundation.
• Changes with a bigger user-facing impact are announced weekly in Tech/News to which anyone can subscribe.
• Quarterly Wikimedia Foundation department check-ins focus on the big picture and progress of annual plan goals and other large projects.
• Many teams have a monthly or sometimes weekly newsletter and/or on-wiki updates page.
• The Wikimedia Foundation Technology department lists its annual plans and quarterly goals on their wiki page, the Wikimedia Foundation Audiences department lists its goals and annual roadmap on their wiki page.
• There are regular Research showcases. Previously, there were monthly Tech Talks.
• People write blog posts about larger or more interesting developments on the Phabricator blog and the WMF blog.
• If you are sufficiently interested in a specific team, team Phabricator boards are public.

For a Wikimedia Foundation example, see PM'ing in public in which Josh explains the steps performed when planning and working on a new software feature.

How I can influence what is worked on?

• Write Phabricator task descriptions and comments as clearly as you can. Provide links to examples. Provide links to related documentation and/or discussions. Summarize the details as clearly as you can, and indicate what is still unknown. Explain the scope of the problem so that the developers can determine how it fits into their other plans. Be friendly/polite. I.e. Make it easy for someone to help you.
  • For reporting software bugs: see How to report a bug for more tips.
  • For requesting new software features: Clearly explain the general type of task that you hope the feature will solve (the actual underlying problem) along with a few specific examples; but do not demand a specific solution, as there might be other/better solutions. Provide links or notes on any related software.
• For software deployed on Wikimedia web sites, the Developers/Maintainers wiki page lists teams and individuals who are responsible per project. Most teams list their communication channels on their wiki pages which allow you to engage, discuss and influence what is being worked on.
• Take part in on-wiki consultations to influence decisions about major software features or new products.
• The Community Wishlist Survey and its voting results help developer teams identify and prioritize popular technical requests for smaller projects.
• Provide a software patch that offers the functionality you'd like to see. If you plan to work on a larger software project yourself, Grants are available to fund community members.

Even if you provided a software patch, review is required before deployment, so the priority is still relevant.

Why are there so many open tasks?

In theory and in the long run, people and time are unlimited, as more people could start contributing to a project at any time. If there is no good reason to decline ("implementing a particular task would be a bad idea") a task, a task may remain open as a drive-by contributor could at any time decide to work on it. However, in practice and in the short run, both people and time are limited resources. Furthermore, it is hard to identify abandoned and unmaintained software projects. Basically anyone can create a task. Altogether this often means that the number of open tasks increases over time.

There were ~45,000 open tasks as of the February 2021 Phabricator statistics. Here is how that compares to various other free and open source projects:

• Drupal has 195,000 open tasks [ref]
• Ubuntu has 135,000 open tasks [ref]
• Chromium has 60,000 open tasks [ref] (out of 940,000 ever filed), and another 50,000 were auto-archived after a year of inactivity.
• Firefox has over 10,000+ open tasks (their search won’t show any more than that [ref], but it was 44,000+ in 2011)
• WordPress reportedly has 6,500 open tasks
• VLC has 4,200 open tasks [ref]
• GIMP has 2,700 open tasks [ref]

Why was the priority lowered? This is important

Phabricator and its purposes

Phabricator is where we, active in the Wikimedia movement in various ways, report bugs, create tickets that suggest features and where volunteer developers, affiliate staff and Wikimedia Foundation staff keep track of work they're doing. Track and discuss technical development. That way, it serves several purposes. It's a list of features someone would like to have, and – given the open-source nature of our projects – someone might work on one day. It's a list of features the developers are actively planning to work on and know when. It's a list of suggestions that have yet to be investigated and thought about. It's where we talk about the urgent problems that have surfaced and need to be fixed now because the wikis aren't working. This combination of collection of requests and project management for software development teams can sometimes be confusing, and lead to us talking past each other.

Priority

One of the features of Phabricator is the ability to put a priority on something: low, medium, high, unbreak now. This is probably one of the most often misunderstood parts of Phabricator. As Phabricator/Project management#Setting task priorities points out, priority is not to be confused with importance. An important task can be of low priority. It might just be a signifier of if someone is working on the task or not. Maybe they've taken feasibility into account: yes, this is important, but right now we don't know how to solve it. Quite often this happens: someone suggests a feature or report what they see as a problem, but it's not a bug keeping us from doing our editing work. It's important, they decide, so they set the priority to high instead of leaving it up to the developers to prioritise. Someone else, maybe Wikimedia Foundation staff, comes along, triages the Phabricator ticket, and sets the priority to medium or low. From the community perspective, this is seen as a rejection: no, we disagree, this is not important. From the developer perspective, it could just have been an indication that no one is working on it right now.

Timing

Doesn't "no one is working on this right now" mean that it's considered less important? Well, it means that it's not considered "oh help us this is completely breaking the wikis and we need to drop everything and fix this urgent bug". But for everything else, for the long-term, known problems, there's the fact that everyone is already working on something important. They have made promises. They have made promises to the editors who were interested in the feature, they have made promises internally so that the Foundation can live up to the goals of the movement strategy, they might have made promises to other developers who are dependent on them, waiting for them to be done with the thing they have said they will do so they can build their thing. The developers behind one feature might need help from others – for example, they want to fix one feature but depend on infrastructure work they can't do themselves – and have to adapt their timing. Sometimes, the work will be done later, in combination with other tasks from the same area; the goal there is to gain in efficiency while improving the code.

So quite often, they look at a problem, think "yeah, this seems important, we need to talk about this". And then they set the priority to medium or low.

Because they are already working on something else, and that's important too.

What can we do?

So what can we do to stop talking past each other? Editors, don't read too much into the priority setting. If someone sets it to "low", it means they may have no plans of working on it, but "low" or "medium" doesn't in itself mean that. If in doubt, feel free to ask in the ticket. And don't set a priority for a task you're not planning to work on yourself.

Software teams and other developers: please do consider taking a few seconds to explain what you're doing. It doesn't need to be a long explanation, just a reminder of what the priority function means, because this is not obvious and understanding it as an indicator of importance is not strange. Do link to Phabricator documentation if it makes sense, but politely explain – as you're lowering the priority – in the ticket itself that this is not because it's unimportant, but that it falsely gives the impression that someone is currently handling it, and that you or your team might not have had time to properly investigate it yet to even form an understanding of what needs to be done and when that could happen.

See also

• How and when accepted ("merged") code changes become available on Wikimedia wikis (in short: usually once a week; sometimes faster via SWAT)
• Code stewardship reviews
• Product principles
• Just make it a user preference