Topic on Talk:WMF product development process/Archive 3

Could someone please clarify what the WMF's vision is here, for community input prior to the build phase?

The Vision is being defined as we speak, and this is the place to define it (or maybe a specific page focusing on community participation in the development process?) Let me try to capture our state of mind in the discussions that lead to this draft:

• Communities have a simple way to watch projects going through stages, resting assured that they won't miss any important step. To be defined and implemented.
• Communities have a say during each stage. What feedback is expected and how it is provided, needs to be defined. This is an open source software development process, and therefore FLOSS and standard engineering practices are likely to be more efficient than Wikimedia's tradition in wiki management. Contributing must / should / could criteria in the Concept phase and identifying blockers before / during the Build phase is more powerful and effective than providing a first and final community resolution in the form of a support/oppose RfC when a product is basically ready to be deployed.
• If everybody has done their homework up to the Build phase, the Release phase should be a simple matter of going through an agreed list of remaining blockers and quality criteria.
• The role of Beta features, A/B tests, user opt-in / opt-out needs to be defined, the idea being that all these ways to test with real users and soft release inform the developers and stakeholders about the status of a new product. Having solved "philosophical" discussions between the Concept and Design stage should allow to proceed with all these test releases with a shared interest from the communities in gathering new data and feedback.
• How soon and how exactly a community receives a new product depends on the specific plan for beta features / A/B tests / opt-in / opt-out / deployment phases... We need to organize these elements in a clear and predictable way. Communities should have a way to manifest their interest in being among the first ones, or their preference to be included in a later wave, when technical and social details got more time to be polished. Deployments should follow paths of least resistance whenever possible.
• Communities have the final say on the deployment of a product in their wiki. This is a right and a responsibility. A community blocking the deployment of a product as planned is expected to provide proof of community consensus and actionable blockers, to be open to discuss the blockers in product terms, and to be involved in their resolution. The aim to provide a consistent user experience across Wikimedia projects should always prevail. Moving from a deployment wave to another should not be a big deal.
• If the team in charge of the deployment of a product suspects that the blocking in a project is not fair, they will report the situation. Where and to whom is to be defined, but it must be a current or an adapted Wikimedia community process, like reporting to the administrators board of a project, the stewards board... An example of unfair blocking would be a single active admin blocking a new product in a small project based on personal taste, without community discussion.

In order to participate effectively as software partners, the WMF teams need to change in some way and learn to interact better, but this is also true for the communities. Non-technical contributors need to understand the basics of the product development process and their role in it. Communities need to understand that they are not the only stakeholder, and that "The Community" is a very complex stakeholder difficult to read due to the many communities and the many voices in each community (including those voices that follow but don't speak because of many structural and occasional factors). The proactive involvement of tech ambassadors, tech-curious contributors, and administrators is essential in order to keep a healthy and productive collaboration.

There are other ideas about systemic bias in community discussions, respectful communication, opportunities for wider segments of the communities to provide feedback in simple/private/non-demanding ways... But this post is getting long, and I hope that at least it is useful to start defining The Vision, and the homework that we need to do in order to implement it.

PS: er... I got carried away and went through the process until deployment, not just up to Build. There are many discussion that can start from here anyway, so I hope you don't mind.  :)

Thanx Qgil. There's a lot to like in that "state of mind". The WMF seems to have thoroughly discussed and documented many of the more... mechanical aspects of the process... but I wish the working-as-partners aspects would be better addressed. Sometimes partners disagree, and sometimes it takes effort to sort out those disagreements. I don't want to rehash old problems, I don't want to assume the worst, but I also don't want to assume the best. I'm hoping for clarity on some points that are still "to be defined".

Individuals, admins, wiki community consensus, global community consensus:

1. Individuals: The WMF can shoo away bad ideas by saying "come back with a consensus".
2. Admins: Admins are individuals. "Come back with a consensus" is perfectly appropriate.
   • Regarding an admin who acts without consensus to block a new product: An admin who disregards consensus is not just going against WMF, that is an abuse against Community. I will firmly support nuking his admin bit, even if I agreed with what he wanted. Admin tools are to be used on behalf of community and consensus.
3. Wiki-consensus: On EnWiki central community consensus is established at Village Pump. I assume other wikis have similar central community pages. If a consensus is asserted elsewhere, it is appropriate and often advisable for the WMF to say "come back with a central community consensus".
   • It is absolutely reasonable for the WMF to request/expect elevated levels of advertisement for important RFC's. It's not reasonable to reject a consensus merely because someone imagines a "silent majority" would have given an answer more to their liking.
4. Global community consensus: For matters that affect the global community it may be reasonable to say that evidence of broader consensus is needed, especially when dealing with a smaller wiki. However that is a rather thin argument regarding EnWiki. EnWiki is about 44% of the active editor community.
   • The WMF deployed Single User Logon relatively recently, the brand new translation extension looks rather successful, cross-wiki notifications are in the works, and it looks like Community Tech team might pick up cross-wiki transclusions (or similar) as a new project. With these tools the community may be able to solve the cross-community complexities for you.

• Contributing must / should / could criteria in the Concept phase and identifying blockers before / during the Build phase is more powerful and effective

I particularly asked about pre-build because it's the best and most painfree stage to address issues. To quote something the WMF said over two years ago:

Flow/Community_engagement#What_we.27ve_learnt_so_far Bringing users into the conversation at the end of the development process makes it hard for their feedback to be incorporated. Ideas gather inertia, particularly when implemented, so it's hard to change course.

In the best case the WMF_product_development_process acknowledges that the community might want a significant change in course, and it acknowledges that "expected community feedback" could be a consensus that a product would be unwanted or detrimental.

To be clear myself, my concept here is dialog. The WMF announces a concept, and in most cases the community thinks it's good or great. *IF* there are community concerns and if consensus is shown, the WMF could either agree, or we need more dialog to get on the same page. If partners disagree on something, both sides probably have good reasons.

Is WMF's idea of "expected community feedback" merely bugreports and feature requests? Or does it contemplate the possibility of feedback opposed to the project itself?

• The aim to provide a consistent user experience across Wikimedia projects should always prevail.

Limiting support to a single configuration can be an absolutely legitimate technical decision, and it absolutely is a valid basis to override minority resistance at the end. However if the WMF follows the "path of least resistance" with initial deployment to small wikis (some of which never seem to object to anything), then hits consensus community rejection, imposing a single configuration would mean rolling back the initial deployment to the first 10% percent of editors. "Consistency" isn't a legitimate basis to force deployment against-consensus to the remaining 90%.

• A community blocking the deployment of a product as planned is expected to provide proof of community consensus and actionable blockers

Proof of consensus: absolutely agreed.

Actionable blockers: Objection. Setting the system font to ComicSans would have a serious negative impact on reader's perception of Wikipedia as a credible source of information. There is no actionable blocker on that. No list of bugreports or feature enhancements will fix it. Can you please confirm that an objection can be both valid and non-actionable? A legitimate issue can't be disregarded simply because it is non-fixable.

• There are other ideas about systemic bias in community discussions

That would be an interesting topic. I would be happy to contribute some ideas about systematic bias at WMF as well. Grin.

In any case, we share the same mission. We're all on the same team. We will reach much better outcomes if we work as partners.

It looks like we start needing an own page to draft and discuss these steps in more detail. Your opinion is welcomed at Topic:Ssjptg7hot8kc3k6.

Is WMF's idea of "expected community feedback" merely bugreports and feature requests? Or does it contemplate the possibility of feedback opposed to the project itself?

In my opinion, opposition to an entire project should be possible, but ideally it would appear at the Concept / Plan phases, before spending lots of donors money developing a project in a certain direction. It should come with constructive arguments and actionable proposals, the same expectations put on any stakeholder during the product development process.

The constructive arguments should reply questions like Do we agree on the problem but not on the planned solution? Do we actually disagree on the problem itself? Is the plan correct but not its prioritization because something else should be addressed first?

Actionable proposals should provide a way forward to resolve the problem as good software partners. From the WMF point of view, if one community is objecting radically to a new product with a solid argumentation, it is probable that at least some of that argumentation will be echoed by other communities and by other stakeholders at the WMF. In that case, changes to the plan should we logical and welcomed.

Can you please confirm that an objection can be both valid and non-actionable? A legitimate issue can't be disregarded simply because it is non-fixable.

If a community finds that Comic Sans in a new product is a horrible choice, then the blocker is "Remove Comic Sans from Product X". I believe all valid and legitimate blockers can be made actionable? The point is to avoid the risk of features being blocked with subjective or collateral arguments without any practical way forward, leaving the developer team with no margin to discuss or improve anything.

systematic bias at WMF

Sure, nobody is exempt of the risk of systemic and systematic bias. If trust, common sense, and healthy partnership prevails, then telling each other how to be a better partner becomes a lot easier. We need to allow to ourselves flexibility and common understanding to put this process in practice progressively, before having defined and demonstrated every aspect for it, improving it whenever something didn't go as expected.