Jump to content

Topic on User talk:Qgil-WMF

A test of the Technical Collaboration Guideline

13
Alsee (talkcontribs)

In the Metawiki proposal to remove Flow, you suggested treating it as a test for the TCG.

I don't think that case is really much use as an effective test. I think the collaboration guideline is about to face a real test. I wanted to let you know in advance, and I invite any thoughts you have on it.

The situation:

The WMF decided to build a new Wikitext Editor. It is currently available as an opt-in beta test feature. According to the plan, in a few months it would become the default wikitext editor. The current editor would temporarily remain as an opt-in option, but eventually it would be removed.

There are currently a big pile of bugs on the project. And of course that's normal and fine. I'm not concerned about them. I have full confidence that our devs will do an excellent job addressing the routine issues. I'm also confident that the WMF isn't going to try to move it out of beta until those routine issue are resolved to a high quality.

However I believe there are two "non-routine" issues. Several editors have told the WMF that these are significant issues. 100% of the (non-staff) comments on the issues agree that they are significant issues. Most of those comments explicitly identified the issues as severe enough to disqualify the project from deployment as the default editor.

I opened a topic on each issue, each topic explicitly stating that the issue should be categorized as an actionable blocker. That statement was in the body of the comment. In retrospect, I probably should have put that in the title of the topic, and I probably should have explicitly mentioned that they were intended as TCG-actionable-blockers. Staff on the project do not seem to have made the connection, and they do not seem to have placed any special significance on them.

Staff have acknowledged the basic validity of the concerns. However the two issues would be either very difficult, or virtually impossible to resolve, given the fundamental plan and design of the project. The response of the staff has basically amounted to "oh well, we can't/won't fix them, and we don't consider them very important".

This is where things always run into trouble. The WMF is great at bugfixes and enhancement requests. But when an issue can't be resolved within the pre-defined project plan, staff deal with it as politely as they can and they just go back to work. And if it's just one person with a dumb issue, that's fine. But then a second editor says this issue disqualifies the project from deployment, and they are politely dismissed. Then the third editor is politely dismissed. Then the fourth editor is politely dismissed. And the project just keeps rolling forwards. Then phases like "talking to a brick wall" start to become common vocabulary in the community.

When you have random individuals showing up with assorted comments, I understand it can be hard to sort out the important issues from the unimportant-individual-fluff. I think part of the problem here is that, for most staff, the only time they encounter an RFC is when it looks like a giant catastrophic meteor falling from the sky. In the community, RFCs are an important, useful, and routine tool. If someone posts on a project page saying they think a project shouldn't be deployed because (issue), I think it would be great if the response was something like this:

That's a hard issue. I don't think that's something we can/will fix. But we will certainly escalate the issue if you point us to a community consensus saying it's really that severe.

Result: The person knows they were heard. They were offered a constructive avenue to pursue their concern. And they have effectively been told to go away.... in the most positive and polite possible meaning of that phrase. Either they drop the non-issue, or the community shoots down their non-issue, or they come back with a consensus saying yes, this really is something important.

The last thing most staff want to do is invite someone to run an RFC, but it really is an excellent way to deal with the community. If the person is a pest ranting about a non-issue, the community will deal with them. If the person is genuinely representing the community view, if the issue really is as important as they're claiming, then that is an issue the project team needs to deal with whether they like it or not. Effectively inviting an RFC in the early stages is far better than a meteor-from-the-sky at deploy phase.

I got rather sidetracked in the last few paragraphs.... but I think it was a good sidetrack. Anyway, back to the two Actionable Blocker issues at hand:

Issue 1: Previews

The New Wikitext Editor does not provide genuine article previews. The "new editor" is actually being designed as a new mode within the Visual Editor extension. The "new editor" actually uses the VE software-stack to generate previews. This means that there's a vast number of things that render differently in the preview than when the page is actually saved.

Project team's perspective (as I understand it)

There have always been all sorts of things that don't display correctly in VE, but it's never been a major issue before so long as it gets most things right. VE is awesome, it's the Future Of Wikipedia, it is the long term plan, everything should move in that direction.

Community perspective (as I understand it)

VE is a secondary editor, used by almost no one. (About 95% of all edits are wikitext edits.) VE's goal was to make editing easier and bring in new people. VE failed that goal. The WMF's own research shows that VE helped an additional 0% of new users to successfully make a first edit. The research also shows it produced a 0% increase in editor retention, and 0% increase in total contributions. Almost everyone uses wikitext because it's simply a better tool for the job.

No one complains about rendering issues in VE because it's a secondary editor, because we understand VE was a difficult project to build, and because anyone who chooses to use VE is choosing to accept any warts that come with it.

We already have an excellent wikitext editor. It already gives 100% accurate previews. The essential purpose of previews is accuracy. Preview errors will frustrate experienced editors and confuse new editors. Replacing accurate previews with broken previews is an obvious downgrade. The team has stated that some of the rendering errors are CANTFIX/WONTFIX. There are currently a ton of serious rendering issues the team does intend to fix, but even if the team fixes a hundred bugs, even if the errors only show up in rare or weird cases, it's just plain dumb to break previews. Many people won't care if the project is aiming for "99+%" accuracy. The blocker here is that the the WMF stop trying to play whack-a-mole fixing most of the endless preview-bugs. The blocker is to provide genuine article views, using the genuine article view engine. That fixes 100% all at once.

Many people are going to be hostile. The view will be that the WMF is breaking our PRIMARY editor... and people will think it's because the WMF's obsession with VE is becoming pathological.

I know everyone at the WMF has good intentions, but the WMF's perspective and priorities sometimes don't quite parallel the perspectives, priories, and experience of editors. Good intentions tend to fall to the wayside when it seems like things are being broken for bad reasons.

Issue 2: Load time

With the current wikitext editor it only takes three seconds before I can begin editing a large and important article like United States.

With the current wikitext editor it only takes five seconds before I can begin editing our largest article.

The New Wikitext Editor is actually a new mode in VE. It requires far more code, it's a memory hog, and I think(?) that it has to spend time analyzing the wikitext before it can do anything.

With the New Wikitext Editor I have to wait thirty seconds before I can begin editing the United States article.

When I try to edit our largest article with the New Wikitext Editor, I'm stuck sitting here staring at my clock for one hundred and twenty seven seconds before I can begin editing.

I don't know if my time measurements will be typical for other people. However the issue is obvious, even if most people get better times than I do.

The perspectives on this issue will be essentially the same as before. For the VE team, slow load times are an accepted fact. No one ever screamed that it was a fatal problem for VE. For the community, slow load times are simply one of the warts that comes along with VE. If you don't like the VE's load time, then don't use VE.

And here too, people are going to get hostile. An editor that takes 30 seconds to load is BAD, an editor that can take 127 seconds to load is absolutely obscene. People will ask why is the WMF screwing up our primary editor? Some people will feel that the WMF's obsession with VE has become pathological, some people will feel that the WMF's disdain for wikitext is bordering on deliberate sabotage.

I first raised the preview issue three months ago, when the very first prototype was available on the test servers. Someone else first raised the load time issue two months ago. Multiple editors have said these issues BLOCK deployment. I explicitly said they should be categorized as "actionable blockers". So we're at the next step. I am in the middle of drafting an RFC to file a formal block against deployment (subject to finding a resolution, of course). I anticipate the RFC is going to pass overwhelmingly. Based on what I know about the project, I don't think these issues can be fully resolved without substantially scrapping the design.

Qgil-WMF (talkcontribs)

Almost every time that I have attempted to bring the TCG to a conversation about an ongoing project I have surprised / confused / annoyed someone, so I have decided to stop mixing both concepts unless the conversation is happening in a social context over drinks (or similar), as a purely mental and open-minded exercise.  :) I guess you have seen my recent comment about this at Meta:Babel.

If you want to propose blockers for the new wikitext editor, I recommend you to document them as tasks blocking T142523. Maybe the issues you are reporting here have already their own tasks? If not, feel free to create them. That is the best way to assure that they will not be missed by the development team or other users interested in the new wikitext editor, since any steps forward for that product will eventually pass through that funnel. That is also where it will be decided which tasks are considered blockers indeed, and which are considered non-blocking problems, no-problems...

Alsee (talkcontribs)

My electricity dropped out and I need to start the RFC draft from scratch :(

In the mean time, I opened the two Phabs you suggested. T154843 and T154844. I don't have high hopes, but I assume the mention of RFC and potential deployment block will draw attention.

Alsee (talkcontribs)

Update:

It's been 9 days since I submitted the two Phab items. Neither has been accepted, neither has been rejected, no one on the project has addressed either issue in any fashion. I wish I could say I expected a more productive result. :(

I opened the RFC: Proposal_to_submit_blockers_on_replacing_our_wikitext_editor.

Aside from opening the RFC weeks ago, is there anything I should have done differently?

Qgil-WMF (talkcontribs)

Since you submitted the Phabricator items there has been a Wikimedia Developer Summit and a Wikimedia Foundation AllHands meeting that has kept many of us busy. Also, why the urgency?

Alsee (talkcontribs)

Urgency? We've been trying to get a constructive response from the project team for ~15 weeks on previews, and for ~11 weeks about load time.

The WMF is great at bug fixes and enhancement requests, but a certain pattern kicks in when an issue cannot be easily resolved within the predefined plan. For example fixing previews would require a fundamental design change. The issue is raised, saying that it's going to block deployment. It gets a polite and completely non-committal response. "Um yeah.... mumble mumble mumble". I'm not sure if people are trying too hard to be "polite" and not say "no", but they either can't or won't change the core plan and the issue just falls down the memory-hole without a meaningful response. Multiple people try to raise the issue multiple times, saying this will block deployment, and it never gets a "yes" or "no" or any other sort of resolution. The issue just keeps disappearing into the page history. The project train just keeps rolling forwards. The WMF just keeps digging itself into a deeper and deeper hole. More and more $$$ and dev-hours are piled on. Then the WMF is predictably surprised when there are problems at deployment phase. "OMG, we've spent all this time and money, and the community is battling us at the last minute."

I didn't know about the Developer Summit and stuff, and I know 9 days isn't a particularly long time up on Phab. But everyone up to and including JDForrester has known about this issue for months, and has been blowing off the issue for months. The only reason I waited the extra 9 days is because you suggested giving Phabricator a shot. There was no acceptance, no rejection, not a single comment indicating that anyone was evaluating it as a critical issue.

The earlier an issue is caught and addressed, the easier, cheaper, and less stressful it is to resolve. I regret that I didn't run the RFC 10 or more weeks ago. It would have helped the WMF to genuinely take note of the issue before investing so much time and money.

What I'd really like is to reach a place where an RFC isn't viewed as some evil combative thing. Where you don't think I was hasty running the RFC now, where the RFC would have been welcome 10 weeks ago. A place where an RFC has one of two positive outcomes. Either the community shoots down unreasonable objections for you (win), or the RFC brings you important and desired information about the project as early as possible (win).

Qgil-WMF (talkcontribs)

The new wikitext editor is an opt-in beta feature in English Wikipedia. Discussion of potential problems of performance and previews belong to this stage. As far as I am aware, nobody is talking about graduating this beta. This is why I don't share your sense of urgency.

Alsee (talkcontribs)

I don't see the issue as "urgent". I merely see it as overdue.

The worst time to address issues is when perpetual inaction drags out to deployment phase. So far the inaction has dragged out for ~15 weeks, despite repeated attempts by multiple editors to obtain a meaningful response.

The best time to address issues is as early as possible, ideally before any time and money is invested in coding. I tried to do exactly that, before a test version even existed. I was informed that there was no information available and that I had to wait until it was built. So we missed the best opportunity. After a prototype is built, the earlier issues are addressed the less costly and disruptive it is to the development process. If there is a change to the plan, it benefits the WMF to know that as early as possible.

Qgil-WMF (talkcontribs)

I think we are starting to repeat ourselves.

I thank you for submitting your proposals for blockers in Phabricator, connected to the right task. This is the most efficient way to assure that your reports are addressed.

Once the problems are reported... is there a benefit in insisting on them on a daily basis? If your ultimate goal is to help the development team releasing the new software with those problems resolved, then I don't think so.

The team is not inactive at all. You can check their priorities here.

Qgil-WMF (talkcontribs)

Also, can it be that you haven't linked to the Phabricator tasks you opened from your RfC? You got some feedback in both, and people interested in your RfC might be interested in those conversations.

Alsee (talkcontribs)

Thanx. Good idea. I added Phab links to the RFC.

Alsee (talkcontribs)

It looks like I was right, this is a real test case for collaboration.

I assume the team will make a good faith effort to look into the load time issue (task T154843), although I question whether they can realistically reduce the time to be similar to the current editor. But let's set that aside.

Jdforrester has categorized the other consensus-blocker as "invalid" (task T154844). Do you have any suggestion on how we should proceed?

Qgil-WMF (talkcontribs)

That task is still open and the conversation is continuing there.

Reply to "A test of the Technical Collaboration Guideline"