Topic on Talk:Technical Collaboration Guidance/Community decisions

> "...the community will be asked for a new review. If no surprises are found, the deployment will proceed."

I wonder whether this "new review" is intended to be limited to reviewing whether the accepted blocker was solved, or if it is meant to be a completely new review. There are circumstances in which either option would be the most appropriate, but I'm concerned that in the currently written form, it is almost asking community members to propose blockers (and getting them accepted and fixed), reviewing again, moving the goalposts, and repeating the cycle endlessly.

AFAIK the "new review" is for the previously identified blockers, otherwise there is the potential for an endless "review loop" as you mention.

If a software development project got stuck in a cycle of trying to fix a product, which customers declined to buy after each iteration, what would be the expected private sector outcome?

If you're a startup, you might consider pivoting. If you're developing enterprise, you'd consider finding and selling to a new client base.

None of these options matters if you're developing open source software for a non-profit, though. Comparisons to business-based solutions are not generally appropriate, as there is no profit motive.

To clarify: the goal of the software development at the WMF is to better serve the mission. That is the lens through which products are built, and the motivation behind them. This is a completely different motivation than that which serves the private sector, and the two simply cannot be compared because decisions are made differently in that context.

You're making a dangerous error suggesting that non-profit status somehow makes the WMF immune to external reality.

A product is always built with an original belief that it will be desired and valuable. Whether an organization is commercial or non-profit makes no difference. But no person or organization is perfect or omniscient. Sometimes a product is simply a flop. If a product has content-corruption issues as a fundamental design flaw, or unacceptably bad performance issues as a fundamental design flaw, or if the design-team simply misjudged what end-users want and need in a product, a reasonably run company terminates the project and stops wasting additional money on it. If a company stubbornly persists in a product that users don't want, reality eventually punishes that irrational behavior. Money stops coming in, and the mangers, developers, and support staff stop getting paychecks.

The WMF has the luxury of being a charitable organization hosting a website with enormous public good-will (Wikipedia). It has the luxury that there is a disconnect between donors and the actual users of the products. The WMF has significant, but not boundless luxury to waste extensive money developing and pushing a faulty or unwanted design. The WMF has a significant buffer of being able to continue receiving donations while the end-users are the only ones who know the product(s) are faulty or unwanted. However it is a serious error to think that the WMF is somehow immune from the same reality as commercial development projects. If the end users become sufficiently dissatisfied with the product(s), they may inform donors of the situation. At that point the exact same reality applies. Money stops coming in, and the managers, developers, and support staff involved stop getting paychecks. That would be bad. The WMF should operate in a reasonable and rational manner. Good intentions for a product don't matter if real world feedback on a defective or unwanted product is being ignored.

Can we at least agree that the WMF should stop sinking money into a project when there is no dispute that it is overwhelmingly unwanted, AND it has fundamental design flaws that result in content corruption?

The Wikimedia Foundation has removed tools and services that it did not believe were delivering on their goals. Some of them, such as AFT5, were valuable to one type of user, and popular on some projects (e.g., the French Wikipedia, in the case of AFT5), but created burdens on others (oversighters at the English Wikipedia, in the case of AFT5). Others didn't really work out the way they were planned (such as MoodBar, which did not achieve its objectives and so was undeployed rather than taking up experienced editors' time, frustrating new editors, and distracting development from priorities).

In other cases, tools aren't developed enough to be evaluated fairly, or they appeal to only certain groups. If we used the visual editor as an example, a few experienced editors on some wikis still believe that it's undesirable to make this editing environment available to anyone.

However, from another perspective, we'd have to account for these facts:

• As of last quarter, at least 30% of all editors use the visual editor.
• Some new editors with an educational focus, such as librarians and GLAM institutions, have previously refused to edit Wikipedia because they did not want to spend the time necessary to learn wikitext, and they are now using the visual editor exclusively.
• Chapters and other organisations training new Wikipedia editors very frequently comment on how the visual editing mode is crucial in making those experiences a success.
• VisualEditor is the most popular extension to MediaWiki, and it and the underlying rich editor software is being used at many other organizations, including NASA and other institutions that undertake educational work parallel to our own.

I suspect that facts such as these might cause unbiased people to conclude that this software is a desirable project. From the product perspective, although this software prompts some debate among some highly experienced editors, it is generally helpful to and primarily used by people with less experience, whose views and needs the team must also take into consideration.

P.S. You might be interested in my suggestion for research about whether new editors who use the visual editor are less likely to produce speedy-deletion-worthy pages under ORES's draftquality measurement, compared to new editors who use the older wikitext editors. I've been looking at the behavior of some new accounts, and I think it might be a significant predictor of success.

I wasn't referring to VisualEditor. The original VisualEditor RFC consensus was "1. VisualEditor is a good idea in theory": there is clear support for this, most people agree that a Visual Editor is in itself a good idea. The WMF then listened to community concerns, and the WMF successfully improved it such that the community agreed that deployment was beneficial. So the VisualEditor example is, if anything, supportive of my point.

Your initial topic was what should happen after the community filed blockers and the WMF did work trying to improve the product. One option was that that the community may still say there are issues that block deployment. The other option was that the WMF declares the community gets one opportunity to identify problems, the WMF then makes one effort to address those problems, then the WMF terminates engagement and forces out the product.

None of us desire a case where resources are wasted on an "endless cycle" of improvement and rejection. The question is whether we should break such a cycle by terminating such a project as early as possible? (My answer.) Or do we terminate the cycle by terminating engagement and forcing out the product? (The effective result implied by Keegan's response.)

I'm horrified, but not surprised, that we're even discussing this.

I think it is difficult for most end users to accurately determine which projects constituted "wasted resources", and which are worth further investments, especially when the product is in early development. That determination can be difficult even for highly experienced, well-informed people with analytics resources and hundreds of hours to devote to studying the question.

I also want to repeat what's becoming a constant theme: There is more than one community. For example, the German Wikipedia and Commons explicitly encourage a type of username that the English Wikipedia bans. It doesn't make sense to talk about what "the community" wants, as if there were only one community, or as if those communities were always adequately represented by the individuals who happen to participate in any given discussion.

I also want to repeat what's becoming a constant theme: There is more than one community.

I also want to repeat what's becoming a constant theme: If wikis representing a majority of the global community reach consensus on a blocker, that needs to constitute a per se blocker that goes to "discuss" rather than "decline".

So until you and the WMF acknowledge Global Community Consensus, I request that you stop making disingenuous appeals to a lack of Global Community Consensus.

We could probably have a nice philosophical discussion about whether a global community even exists, but it's kind of off topic. What's relevant here is that the TCG says that only the product manager can decide that a problem blocks deployment. That means not "the WMF", and not "the Global Community Consensus", and not anybody else.

I concur that the current text of the TCG contains words saying that the product manager can decide whether a problem blocks deployment.

However, as Qgil has acknowledged in previous discussions about the TCG, that is where processes within the TCG end. The TCG is advice for staff, on best practices for inviting community involvement. The post-TCG process is then likely to proceed with the community taking action which has the actual effect of blocking deployment.

Your move.

I'm having difficulties following the points being discussed here, but since I have been mentioned...  :) The TCG aims to cover the entire software development process, from beginning to end. It also aims to document best practices both for development teams and communities.

"The community taking action" and "blocking deployment" are expressions semantically charged, but I wonder what they mean in practice. Technically speaking, Wikimedia communities may reach consensus on specific topics. Deployments can be pushed or blocked by the people who have the corresponding permissions. In some circumstances, wiki administrators or other users might find ways to boycott a deployment. However, there is no single or simple connection between "community taking action" and "blocking deployment". Just like there is no single or simple connection between i.e. "Wikimedia Foundation proceeding with plans" and "ignoring communities". What we have are different cogs either collaborating efficiently, disconnected, or spinning against each other.

I would have to work to find the link, but you and I were discussing what happens after the WMF declines a blocker. You explicitly acknowledged that the community may respond by following through on consensus. Setting aside semantic trivia whether something "blocks deployment", we are discussing action which has the practical effect of reversing the problem, blocking the problem, disabling the problem, or otherwise attempting to improve the configuration and functioning of the wiki. (Which, from the WMF's point of view, nullifies the intended deployment.)

You said that is outside of the TCG. You said the WMF would not respond with Superprotect or any similar methods. Your approximate words were that we would turn to the "normal wiki dispute resolution process". I was unable to get clarification whether the WMF's view of "normal wiki dispute resolution process" was compatible with the community's understanding of "normal wiki dispute resolution process".

If you want concrete examples, I'll give two.

The WMF gave explicit assurances that VE would not be made the default editor without asking the community first. VE was then set as the default editor. The WMF was completely non-responsive on the issue for almost two weeks. When I escalated the issue to the Executive Director, we were then told it would be reversed. After weeks of inaction, the WMF then stated it would not be changed. The EnWiki community wrote a patch for the sitewide javascript that would override the WMF's setting.

The javascript was never deployed. The WMF did change the default. The discussion here is what would happen if the WMF had declined to roll back the VE-default? What would happen if the community did deploy the javascript?

Second example. The EnWiki community currently has a standing consensus against the NewWikitextEditor. For several months, I have been attempting to get a constructive response from the WMF on how it intends to resolve this situation. The project manager is completely non-responsive, and the liaison is unwilling or unable to constructively address the situation. Time is up on this. I have pretty much decided to just open a second RFC on the NewWikitextEditor. The three RFC options are:

1. Withdraw the previous-RFC blockers. (Extremely unlikely)
2. Reassert the block, assert that the NewWikitextEditor not be deployed to opt-out without a new consensus to do so, and take no concrete action. (Possible)
3. Reassert the block, ask the WMF to roll back the NewWikitextEditor beta, and if necessary take action to do so ourselves. This would involve an editfilter informing users of the planned removal from beta features, possibly followed by an editfilter or javascript block of the NewEditor itself. (Possible)

Yes, that last option is very very ugly. However I literally posted an image of a big red flag warning that we were heading to a potential hot-conflict. The WMF was explicitly warned about that last option. The WMF has been ignoring the situation.

Qgil, when I tried to have an abstract discussion with you this kind of problem, your response was basically that the TCG is so awesome and the WMF will be so collaborative that we're never going to have this problem anymore. Well, here it is.

When I first saw your TGC page with the metaphor-images of the friendly boat visiting ports, my instant reaction was to go to commons and replace the images with a more obvious metaphor. I found a rather impressive series of a train crashing through warnings and barriers, barreling straight at the community, and running us over. The unstoppable train metaphor. I didn't save the edit. This project is the unstoppable train.

For a year, the WMF has known there was a serious problem here. I tried appealing to your TCG, giving the WMG exactly the sort of Actionable Blockers the WMF asked for, with exactly the process and language the WMF wanted. The WMF ignored it. The WMF has no plans for constructively resolving this conflict. The WMF is just driving the train forwards. Whoever is driving the train is either asleep at the wheel, or they explicitly plan to drive this train over the community on deployment day.

I believe we agreed that the last thing we want is for a project to blow up on deployment day. Well, if we have to find out what the "post-TCG-process" looks like, we may as well do it now. What happens if the community says YES, we really are putting up a blocker on the NewWikitextEditor? What happens if the community shuts down the NewWiktextEditor beta?

Yes, I remember that conversation and I haven't changed my mind. I keep thinking that the answer to your "what would happen if..." questions is "normal wiki dispute resolution process", simply meaning whatever dispute resolutions processes exist in our movement for the problem at hand.

Maybe it is a matter of perspective, but it has been a while that I am not seeing trains crashing. Not even the examples you put, the blockers you propose and the many RfCs you start or encourage look to me like trains crashing. We are a very big complex community of communities, having everybody happy always is difficult, knowing what are the right next steps is not always straightforward, and frictions appear from time to time. Still, beyond the small world of Phabricator tasks, RfCs, VillagePumps, specialized Talk pages... there is a big community of volunteers choosing to put their time and creativity in Wikimedia projects, and beyond that there is an even bigger audience of Wikimedia users that trust us, find us useful, and hopefully will come back to us for more. People like you or I discuss in our little corners for the happiness of those volunteers and those users.

From this perspective, I think we are doing fine, and improving.

On the general subject, I think that this is a hard thing to address in policy. On the one hand, there are problems with saying that proposing blockers is a one-time event. To give only one of the obvious cases, what if problem X is identified and fixed, but the "fix" creates an even worse problem? On the other hand, if an individual doesn't want the project (for any reason or no reason), then that editor may be motivated to find an endless list of allegedly critical problems, when the real problem is "I just don't like it" or "This helps other people instead of me" or even "I thought this project was the opposite of what it was".

For large projects, it may make more sense to think of this process as a continuous, overlapping discussion: The proposal, discussion, fix, and review for problem X is completely unrelated to the proposal, discussion, fix, and review for problem Y. But it's less clear how that continuous, overlapping process reaches a single decision point, such as when to move the preferences for a feature out of the Beta Feature tab and into the normal preferences section.

For small projects, a unified proposal/discussion/fix/review process might make a lot of sense.

I don't think that it's useful to fixate on "forcing out the product". That's not actually possible in some cases (e.g., Contributors/Projects/Accessible editing buttons), it is unusual even to have it discussed, and AFAICT it's never been necessary. (For example, in 2013, the English Wikipedia went from starting the formal proposal to hiding the tool in about 18 hours, and it likely would have been unnecessary if they'd just waited until the next day, i.e., when the PM was back in the office.)

On the other hand, if an individual doesn't want the project (for any reason or no reason), then that editor may be motivated to find an endless list of allegedly critical problems

An individual gets squashed by consensus.

I don't think that it's useful to fixate on "forcing out the product".

There would be a lot less focus on that topic if you weren't up to your eyeballs in the issue right now. There would be a lot less focus on that topic if were to acknowledge that there is a standing consensus against the NewWikitext editor on EnWiki, and give an explicit statement that it won't be deployed as the default wikitext editor on EnWiki without community reversal of that consensus. (And any other wiki that may reach a similar consensus.)

An individual doesn't always "get squashed by consensus" at the smaller wikis, or even at poorly attended discussions at the larger wikis. The TCG needs to work for all of the wikis, including the ones that have only a few active participants.

I can solve the 'rogue wiki' problem for you. But you'd have to acknowledge some form or mechanism of Global Community Consensus first.