Topic on User talk:DannyH (WMF)/Flow

Discussion moving to Talk:Collaboration/Workflows

Hi. Could you please clarify the nature of Workflow for me? Is it:

1. Further development of Flow functionality, or
2. Targeted at existing wikitext pages, fully functional without Extension:Flow?

Thanx.

The Workflows are going to need structured discussions, so the current plan is that Workflows will use Flow as a service, in the same way that Flow uses VE.

But the way that Flow uses VE is different from the way it looks and works on article pages -- the toolbar is different, and there are features like the mentions icon and the VE/wikitext switch that don't exist in article page VE. The same thing will inevitably happen with the Flow elements in Workflows -- there will be different use cases, and the look and functionality will be different. Useful things that we develop in Workflows will probably port back to Flow.

Does that make sense? It's a complicated thing to explain.

(PS: I accidentally wrote this message using my personal account -- it should have been DannyH (WMF).)

The community can already:

1. Automate creating an arbitrary new pre-formatted page. The community already made an AFD script that does that.
2. Add arbitrary content at the top of a specified page. The community already made an AFD script that does that.
3. Add specified content at the bottom of a page/list. The community already made an AFD script that does that.
4. RFC's already have arbitrary time-span templates, we have bots that update the RFC itself, which move the RFC off of the open-list, and makes a list of discussions-to-be-closed.

By golly, I think I saw something like that recently..... oh yeah.... that looks a heck of a lot like the mockups for your Workflow builder. The only thing it's missing are the software-enforced metal cages - like trying to enforce some sort of lockdown on who can close.... and putting SUPPORT/OPPOSE buttons on a response we require to allow free-form responses other than support/oppose. Metal-cage structure we don't want.

I'm a programmer, yeah I know why you want to work with structured data. But don't say you can't build a workflow-builder for existing pages, a workflow builder that can replicate stuff like AFD scripts..... the issue is that the WMF doesn't want to build that.

In the last WMF election every newly elected board member ran on a platform that Flow must not be deployed unless the community consensus wants it. There's a lot of stress over the unending WMF-Community bad engagement and conflicts. I don't mean to target you specifically.... but it seems rather obnoxious to me that the WMF's response to the Flow controversy is to declare that they are supposedly putting Flow development on hold and supposedly shifting focus to a project to help the community.... a clearly wanted-project.... which is then deliberately designed to NOT WORK AT ALL on existing pages.

The community on several of the biggest wikis have made scripts and bots for AFD and RFC. But there are some long-term problems with relying on community-written scripts, which we're concerned about.

Using scripts and bots is within the skill set of very experienced contributors, and not necessarily others. As these processes get more complex and more automated, the learning curve gets steeper for the next generation. It's also very difficult for people in other languages besides the top handful to copy processes from the bigger wikis, and figure out how to modify/scale down to get an approach that works for them.

The Workflow Builder wireframe is just a concept to show a direction right now, but if we do end up going that way, the intention is that the community on each wiki gets to choose how that process works. The section in that wireframe for voting has options for whether/what kind of voting that process would involve, or no special voting item at all. The section on who can close can be answered with "administrator" or "anyone", depending on the community and the individual process.

If we go in the Workflow Builder direction, then the point of that builder is to not put a WMF-created wire cage around those options. The idea is that the community would choose. The same is true cross-wiki -- we don't want to put an English WP-created wire cage around Chinese or Portuguese.

Now, there is a very good chance that the Workflow Builder idea won't work -- that's always my assumption with early concept work like that. But we need to do a lot more research and talking and trying to build things before we get to a real solution. An early concept wireframe is a place to start. It is not a mockup or a plan.

About Flow deployment -- I've been here about a year and a half, and we have not deployed Flow in communities that don't want it. We've been working closely with communities that are enthusiastic about Flow, and offering them choices. I don't like to use words like "never" when I'm talking about future plans, because the universe is vast and surprising, but I think the Collaboration team has been reliably acting in good faith in this regard.

I'm sorry if I was unclear. You tried to tell me that workflow needed to be Flow only. I pointed out that the community was capable of building those workflows on existing pages, so obviously the WMF is capable of doing so. (And if not, you should hire the community members behind Twinkle etc. Chuckle.)

At first I thought it would be impossible to implement the bit about "who can close" for our existing pages, but I just figured out how to do it. Anyway, I was saying that is unneeded and (I believe) unwanted.

there are some long-term problems with relying on community-written scripts, which we're concerned about.

Agreed. Build workflow for our current pages and our current editor (and Flow compatible if you like). The mockup looks like a very nice improvement over the community having to kludge scripts to do this stuff. Integrating it properly from your end means that people won't have to muck around installing scripts to get access.

It's also very difficult for people in other languages besides the top handful to copy processes from the bigger wikis, and figure out how to modify/scale down to get an approach that works for them.

As above, Agreed.

I like the project, except that you are making it Flow only and VE only. I can see bits that were missed and valuable enhancements that could be added. I'd be happy to contribute.

From: https://en.wikipedia.org/wiki/Wikipedia_talk:Flow#Priorities_for_the_Collaboration_.28Flow.29_team

Further development on these projects will be driven by the needs expressed by wiki communities.

If I bring you an RFC result expressing the need that workflow be built for existing pages and our current editor, will that do it?

An RfC isn't needed, because we already know that many editors want as little change as possible. What we need to collectively do, and will be doing over the coming months, is clearly establish how many of the existing workflow processes, and the broadly desired improvements and expansions thereof (across all languages and projects), are possible using a plain wikitext database. Wikidata (wikibase) uses a different interface and database system, because of the increased power, structure, and scaling, that they bring.

Big-picture and long-term, there are a lot of features that would make Workflows more useable and extensible, and less fragile: Being able to follow a single discussion on a page, a searchable archive rather than a long series of archive pages, the ability to filter discussions ("discussions about X involving me and User:Foo, from last summer"). Plus - making these processes easier for smaller languages/projects to develop, modify, and maintain; They don't want a duplicate of the Enwiki systems, because those are over-complicated for their current needs, but they want something that scales as they grow.

Things like "per-discussion watchlisting" are immensely difficult with a plain wikitext page, unless we turn each section into a transclusion, which is inherently unscalable (a few wikis have done this at their VillagePumps, and a few wikis do this with processes like AfD - but it is even more confusing for newcomers than the current system if implemented at every discussion page, and it makes implementing things like topic-filters and changeable-topic-orders a cacheing problem).

Using wikitext pages also means: 1) we continue relying upon <dl> and <dd> to create indents, 2) we continue to expect users to comprehend updates to discussions via either re-reading an entire thread to look for new interspersed comments, or making sense of how a diff relates to the discussion as a whole.

Hence, over the next couple of quarters at least, the team is mostly doing research, as well as work on cross-wiki notifications and other echo tech-debt.

Re: "I like the project, except that you are making it Flow only and VE only." VE requires javascript, and ~1% of our editors don't use javascript, and many of our editors currently enjoy or prefer working with wikitext. (Personally, I've dabbled in HTML since the '90s, and I've always done so in a plain texteditor. I prefer sourcecode, but I can totally understand how other people might not.) Flow works with wikitext, and will continue to do so.

Side note: Flow works with wikitext, and will continue to do so.

No it doesn't. I tried copy-pasting a sample of Talk page article work into Flow. Flow exploded.

There's discussion and work around making the "switch" button (currently the "</>" at the bottom right of the editing pane), for switching between VE-mode and wikitext-mode, more intuitive/discoverable, at phab:T101316. If you paste wikitext into the editor whilst using wikitext-mode, it should work as desired. (The last mode that you make a change in, will be saved as your preference for next time.)

I just tested. Using the </> button didn't change anything.

1. Rendering it inside a Flow post mangles it to usability.
2. Serious parsing errors.
3. The editing interface itself goes haywire, and it becomes effectively impossible to do simple text-editing on it.

Well, one minor improvement I guess. The editing interface didn't die until I hit the preview button.

Alsee, I'm not sure why you're seeing so many problems. If this is a bug report, then we need some more information before we can figure out how to help you.

Questions that would help:

• What's the board you were on?
• Can you show me the text that you were trying to post?
• What's the error that you see? "Exploded", "haywire" and "die" aren't giving us enough information. Can you take a screenshot?
• What browser are you using?
• Are you using a non-Vector skin, or any personal css or js that might be interacting badly with something in Flow?

I had tried to submit a bug report, but your team has systematically removed every Talk page for Flow and the Collaboration team and yourself etc.

I know about eating your own dogfood, but you're forcing us to use Flow to contact you at every official contact point. Flow is a really bad communication medium for Flow bug reports, when bug report content can itself cause Flow to crap out.

I'm not sure how well this will work, but I left the disaster-content pasted on the Wikipedia_talk:Flow/Developer_test_page:

https://en.wikipedia.org/wiki/Topic:Sor98w5yvlxnd79o

The original version is extracted from:

https://en.wikipedia.org/wiki/Talk:Fields_Medal/Archive1#RFC

Expected result: Any possible content (including that table) needs to render identical to how it renders on an article page, with the same functionality as on an article page. A Talk page is an article page, merely with a different URL. That inherently provides Talk pages with exact rendering and exactly the same functionality.

Are you using a non-Vector skin, or any personal css or js / Browser

Firefox. Vector skin. I have Twinkle, and I dunno probably one of the other standard items like Twinkle. I'm pretty sure there's nothing weird.

What's the error that you see?

1. (revised) Expand width should to exactly match Article page width, without horizontal scroll bar.
2. Flow mangles the table structure, for example text after the table seems to be inside a table cell.
3. When I click to edit the text at the bottom, the auto-scroll jumps so the cursor and local text are way off screen. I can manually scroll back, but every arrow keypress throws it off screen again. Editing seems to be technically possible, as long as you don't mind not being able to see what you're doing.

P.S. I'm not sure if Quiddy told you, but phab:T112332 and phab:T112338 were filed based on another post I left there. I don't use Flow, I just find multiple problems every time I test.

I'm not sure if this counts as "bug" or "design", but Flow handling of : * # (indent, bullet point, list), was surprising and crippled, compared to expectations. Indentation is huge, no nesting support.

Quiddity looked into this, and it looks like this is a VE bug that Flow has inherited, because Flow uses VE. Copying raw wikitext into a wikitext editor works fine; the problem happens when you copy from the rendered page into the VE editor.

This diff shows the same bug happening in a VE page, not using Flow:

There's a ticket filed with the VE and Parsoid team: T112394

Copying a Table from a Wikipage into either mode of Flow fails. https://en.wikipedia.org/wiki/Topic:Soz4fg3ci7tkpaz7

The rendering of indents and list types, appears to be identical to what we see in regular wikipages, if we use wikitext. Compare w:en:Help:List with w:en:Topic:Soxgwu1j8bubl9w9. Or see this screenshot. All valid list wikitext will work, when pasted into either the wikitext-mode or the visualeditor-mode. The indentation is normal and nesting support is complete.

The only times I've seen errors with combinations of : * # were when:

1. the editor accidentally added spaces in between the characters, in wikitext-mode,
2. the editor tried to manually type complex wikitext-markup in the visualeditor-mode.

   (VE has some support for auto-converting some basic wikitext-markup (which was added after a lot of editor-requests for it), which can give a false sense of security. However, there is no obvious solution to what should happen when someone types "* 1 " into a wysiwyg editor... are they typing a numbered list, or the start of a number? I know googlegdocs also doesn't handle this well. It's a hard problem, for richtext editors.)

Note: if we type : into any visualeditor interface, it will result in a <blockquote></blockquote> - this is discussed in phab:T71689 and is unrelated to Flow. It does make sense, and is far better than mis-using <dt> & <dd>, but change is difficult.

It seems I was unclear about :*#. I was trying to cite the kludge of having two incompatible semantics in the same message system.

Foo. Alternate Flow mode.

Colon = Small indent.

• Colon-star = nested.
  1. Colon-star-hash = nested.

• Star = Bullet.

1. Hash = List

Foo. Default Flow mode.

Colon = Large indent. Indent size is not equal.

*Colon-star = No nesting support(?)

*# Colon-star hash = No nesting support(?)

• Star-space = Bullet. Star-space does not equal Star.

1. Hash-space = list. Hash-space does not equal hash.

That's not simpler. That's not easier to learn. That is insanity.

As a programmer, as a user, it is an unholy kludge having two incompatible semantics in the same message system.

And for fun lets try various copy-paste combination and see the chaos we get:

Test 1:

Foo.

Colon

• Colon star
  1. Colon star hash

• Star

1. Hash

Test 2:

Foo. :Colon :*Colon star :*#Colon star hash

*Star #Hash

Test 3: Foo.

   Colon

       Colon star
           Colon star hash

   Star

   Hash

Test 4: Foo.:Colon:*Colon star:*#Colon star hash

• Star#Hash

Test 5: (WTF, Flow wouldn't even let me copy more than this part)

Foo.

Colon

• Colon star
  1. Colon star hash

Test 6:

Foo.

   Colon

       Colon star
           Colon star hash

The `:` inconsistency, now filed as phab:T112690.

I believe the other inconsistencies are based on editor-request - I.e. many requests for shortcuts (e.g. star+space) that match basic usage in other rich-text software, particularly those that overlap with wikitext. I will look/ask around for more details.

Please correct me if I have misunderstood the situation:

1. DannyH (WMF) has withdrawn from from his own page(*), and the discussion has been handed off to the Community Liaison.
2. The answer to my question appears to be "No".

(*)Footnote: I had this happen before. A Project Manager withdrew from his own Talk page, and the Liaison's behavior was outrageous. If I asked a question AND said something else, the Liaison replied to the other text and wouldn't even acknowledge I asked the question. I resorted to stripping down my post to the point where there was zero distracting text that the Liaison could divert to. I stripped down my text, so the only way to not-acknowledge I asked the question was to not-reply at all. And that is exactly what the Liaison did. The Liaison simply stopped responding to multiple direct pings. I asked at least a half dozen times.

Quiddity, I know you're miles better than the other Liaison. But I hope you can understand my feelings here. If the answer to my question is "No", it would avoid enormous frustration if DannyH would simply post a "No". There is a level of respect in honest disagreement.

Alsee, I haven't retreated. You've been asking challenging questions, and I'm taking them seriously. After I read your message, I talked to Nick about the best way to explain what's currently going on with the project. He said smarter things than I did in that conversation, and rather than copy it from him and post it under my name, I told him to go ahead and post it.

Essentially, this discussion that you've started is more about the product direction than it is about me personally. You posted your original question on my talk page because you wanted to make sure that I saw it and responded. But I've been thinking about this as a "Talk:Flow" discussion that happens to be on my talk page, and Nick and I both post on discussions like that.

I'm sorry about my rapid frustration level. It gets really hard when it looks like another repeat of the same old problems.

On one level this is about product direction. But on a higher level it's about WMF-Community engagement. The details of any particular project are trivial compared to the WMF-Community relationship itself. On the WMF page for improving Community Engagement, one of the exact points I raised was an inability to change product direction. In too many cases the WMF has built a beta, simply announced it's being deployed, then invited individuals to suggest bells and whistles to hang on it. That is an invitation for fail and conflict. Fortunately this project was announced pre-build, when it's possible for the Community to raise problems and issues and direction in a relatively pain-free stage.

I came to your page, and this is about you, in so far as what you think WMF-Community engagement means. You said the project would be driven by the needs expressed by the community. When I asked about that, I was immediately told that the Community were Luddites, that an RfC "isn't needed", and apparently(?) it would be ignored.

Maybe I should clarify. Any random yahoo can sign up and make an account here. *I* am a random yahoo. It's fine for the WMF to put up a suggestionbox where random individuals can submit random ideas. And it's perfectly fine for the WMF to casually disregard those individuals. But none of that is engagement with the (proper-noun) Community. A formal Central Community Consensus is the voice of the Community.

I'm fine with you ignoring my dumb ideas, but there is a problem when the WMF doesn't consider the Community to be a partner.

1. I think the current project direction is wrong.
2. I think the Community will think the current project direction is wrong.
3. I suspect you concur on #2. (Feel free to correct me on that.)

So.... I'll step back and ask: What do you think is an appropriate next step?

I don't concur on #2, because I don't think that there's a single Community. There are many projects and languages, and they have their own needs and their own voices. I didn't say "the needs expressed by the community," I said "the needs expressed by wiki communities."

There are several languages and wikis that are now using Flow on user talk pages and wiki discussion spaces; they're enthusiastic about the project, and asking for more features. The focus for Flow development over the last year has been to build features and make changes requested by the communities that are using or plan to use Flow. That can mean that the largest few wikis are not currently served by the product; we've been focusing on the wikis where we're providing value to the movement as a whole.

Danny, that is almost exactly the response I was expecting. So I'll simply move to the third level, and I will just ask again.

First level, I expressed my personal view.

Second level, I asked If I bring you an RFC result expressing the need that workflow be built for existing pages and our current editor, will that do it? And yes, I had EnWiki RFC in mind.

So now I just move to the third level: If I bring you a multiple RFCs and/or cross wiki RFC, reasonably representing a larger community consensus, will you respect that?

Note: I had to edit this post because copy-paste mangled things again.

Ping DannyH (WMF), any response? It's been 4 days, and I see you've been active editing on three of them.

17 days, no response?

Edit to summarize: We agree on lots of stuff. The one point of disagreement is that you do not want to support existing pages. I'm asking if you will build support for existing pages, I'm asking if a multi-communities consensus asking for it would be enough to get you to build it?

Alsee, I can't really answer a hypothetical question like that. You should do whatever you feel like you need to do.

Hi Alsee. The recent announcement says, "Further feature development on Flow discussion tools will be based on an assessment following the 'workflows' research work, and on editor feedback at the wikis already using Flow." I.e. They're already planning on doing a reassessment of Flow, a few months from now. I think that reassessment will help (everyone) a lot, especially if we have the benefit of a lot more research and details all collated together. I hope you'll consider waiting a few months, and work with me/us to create something inclusive and thorough.

In more detail, and again trying to write to you as a fellow-editor (I.e. Mostly with my "as an editor..." hat on, but without changing accounts because that is even more confusing…): I'm worried that because some of your expressed perspectives are towards one end of the spectrum (e.g. the hostility implied behind your phrasing of "a bunch of change averse Luddites" as a characterization of what I wrote above about many (not all or most, but many) editors wanting as little change as possible), this is making this discussion more adversarial than either of us want it to be. So, here are some more thoughts to elaborate and hopefully clarify, because as you wrote, we do agree on a lot of stuff...

I personally dislike most general website redesigns, and most replacements/overhauls of things that I rely on. I still personally prefer wikitext editing for most things (except wikitables!), and do all my personal website editing in a notepad-equivalent (gedit), because I like to see and try to understand how it all (all!) works. I personally want many current aspects of Flow to change over time, or to have alternative options (e.g. infinitescroll vs pagination, which non-javascript users already get so should be feasible).

But, I also want to stop excluding so many of the potential-editors who don't want to have to learn the secret-handshake of ::::*. A few people have advocated the position that we shouldn't even encourage newcomers, if they aren't willing to learn all the nuances and complexity of wikitext [in various VE discussions]; I think that's a valid (albeit extreme) perspective, but it ought to be balanced with all the other perspectives.

I also want smaller wikis to not require a large group of reliable bot owners and javascript/template authors just to get basic automation of workflow processes. I want to help existing editors, bot owners, and javascript gurus, do work on less routine/mundane tasks. I want to know what is possible (and what we'd lose), with a well-structured system and consistent APIs and UIs for accessing/manipulating workflows & discussions.

This is similar to what Wikidata does, and what Wiktionary (and I think Wikispecies?) has been trying to do for over 10 years - cf. WiktionaryZ, UltimateWiktionary, OmegaWiki, and the latest (and hopefully eventually successful) Wikidata:Wiktionary proposals - take the optimal bits of standard mediawiki, and add or adapt certain pieces. Which bits they want to keep or change, and how exactly they do it, they have been learning and changing over time. They often start off with very experimental features, and then reduce or expand those features based on various factors. The future reassessment of Flow ought to take into account all the possibilities, including the things that could be changed/improved vs the current version of Flow, as well as the things that could be changed/improved in wikitext. Eventualism is the philosophy that built much of our projects, and it applies to software as much as to articles and other components of our work.

So, please, would you be willing to wait until we (collectively) have the time to do it slowly and right, rather than right-now?

I'm worried that because some of your expressed perspectives are towards one end of the spectrum (e.g. the hostility implied behind your phrasing of "a bunch of change averse Luddites" as a characterization of what I wrote above about many (not all or most, but many) editors wanting as little change as possible)

Perhaps you don't realize how offensive and adversarial your comment was.

I am seeking improved WMF-Community relations. I was asking what the response would be to an RfC outcome. You weren't characterizing "some" editors as change averse, you were justifying a disregard for Community Consensus, and the Community itself.

Your latest post also (politely) implied that I had extremist views. Lets be very clear what's happening here.... I repeatedly and explicitly asked how Danny/WMF would respond to a Community Consensus result. I explicitly took my own views out of the equation. I explicitly said it was fine for the WMF to ignore any dumb ideas *I* might have. The issue here is whether or not the WMF is willing to constructively engage the Community.

We are, and intend to continue, working constructively with as many individuals and groups from the communities as we can and are interested. The results of an RfC (that you wrote) might be interesting, but we obviously can't promise (which seems to be what you're asking) that the results would determine the prioritization of work.

I'm pleased to announce that the Community Tech team has a new Product Manager – Danny Horn, who will be moving from the Collaboration team to join this new initiative.... Danny’s role on the Collaboration team will be filled by a new Product Manager which the Foundation is hiring for now.

Danny/Quiddy, could one of you clarify whether Danny is still in charge of workflow, is someone else temporarily in authority over the project, or is it currently in limbo under a vacant position?

I've moved to Community Tech, so I'm not working on workflows now. The plan is for Collaboration to hire a new product manager, and meanwhile, the other leads on the team are in charge -- Roan, Pau and Quiddity.

As I said in this thread about a month ago: "We need to do a lot more research and talking and trying to build things before we get to a real solution. An early concept wireframe is a place to start. It is not a mockup or a plan." That's still the approach, as Quiddity described it above.

That process will probably go slower, because they have to hire a product manager, and that is product manager-heavy work. But that's what the team is doing.

Could you (or someone) create a Workflow page? That way I don't have to guess where this discussion should be continued, and so it doesn't get fragmented on on the Talk page of a temporary project lead?

That sounds like a great idea. I'll close this conversation, and you can continue your debate anywhere you like.