Following up from invite. I'd suggest at the minimum if a new paragraph is created that has no citation included.
Topic on Talk:Editing team/Community Conversations/Flow
@Xaosflux, a follow up message to say: thank you for sharing the above.
The criteria you shared (new paragraph + no citations) is precisely how we're planning to configure the "Should Edit Check get activated?" logic, initially.
Over time, we're thinking that projects will be able to make this logic more "robust" (read: minimize false negatives) by configuring Edit Check in ways similar to what you described below.
Note: the project page contains a bit more context about how we converged on the above which I suspect will look familiar seeing as we (you and the Editing Team) seemed to have "landed" in the same place.
Excellent – thank you for sharing this idea, @Xaosflux. I've added it to the ticket where we're defining the initial heuristic.
I've emphasized "initial" in the sentence I wrote above because we (the Editing Team, and folks like @Sdkb, @Mathglot, @Phlsph7, @Elmidae, and others) have been thinking of it being crucial[1][3] that we be able to iterate upon the heuristic this initial "reference check" will depend on (and potentially others in the future) together.
With this in mind, what information/abilities can you imagine yourself needing to have access to in order to evaluate the effectiveness of this "reference check"?
A couple of ideas that we've been thinking about...
On a per-project basis, volunteers ought to be able to: (1) see the rules that determine when "edit check" is initiated, (2) see the edits the current configuration of "edit check" rules would cause the system to flag, and in the future, (3) adjust the rules mentioned in (1).
Having rules live on-wiki (like at MediaWiki:EditCheckRules.json or the like) would help projects adjust to their needs in a lighter weight format than the abusefilter (inspired by the newcomer tasks setup). That would satisfy 1 and 3; as far as 2 - not sure. Actual edits with it should get identified (a revision tag would prob suffice). It seems like edit-check is going to interfere in the publish workflow - this should be carefully considered with how it overlaps or times with ORES and AF - there are certainly edit types that could trigger all 3 of these things.
Having rules live on-wiki (like at MediaWiki:EditCheckRules.json or the like) would help projects adjust to their needs in a lighter weight format than the abusefilter (inspired by the newcomer tasks setup)
@Xaosflux: well put and agreed. I've added what you've suggested above to T327959#8725987. This is the task where we'll need to work together to figure out how to go about implementing Edit Check's on-wiki configuration.
...as far as 2 - not sure. Actual edits with it should get identified (a revision tag would prob suffice).
Noted. It sounds like we're on the same page...
The Editing Team is planning to do as you suggested above: introducing a hidden revision tag (T324733) that will enable us (volunteers and members of the Editing Team) to review the edits (T324734) the Edit Check heuristic (T324730), as currently implemented, would identify as needing references.
It seems like edit-check is going to interfere in the publish workflow - this should be carefully considered with how it overlaps or times with ORES and AF - there are certainly edit types that could trigger all 3 of these things.
Great spot...
Do you happen to have a scenario in mind where you think what you described above could come to fruition? I'd like to make sure I have a clear image of what you're seeing in mind so that we can account for it and if possible, ensure people do not experience a deluge of potentially redundant messages.
Xaosflux, in practice, how many of the Wikipedias do you think would be able to manage a JSON file locally?
Any if it was guided such as this one: testwiki:Special:EditGrowthConfig --> testwiki:MediaWiki:GrowthExperimentsConfig.json
That would limit the tool to "pre-built" modules. You couldn't have one that says, e.g., that the local community has decided on a particular style for writing about Covid/covid/COVID/COVID-19. I think that would probably be okay, but it would be a limitation.
So instead someone is going to write this in to php for every topic a community wants? Being able to empower self-service somehow seems better; if some sort of "trigger" needs to happen that could also be in a config file, no?
Hmm. On the one hand, implementing a very flexible, AbuseFIlter-like system inside the editing environment would maximize local customizability. On the other hand, the existing AbuseFilter can't be used in practice at most Wikipedias (too complicated for average admins), and its use at some wikis, especially the wikis that have only one or two people who can write regex, can cause problems. Poorly tested filters can have unintended consequences, even when the intention is noble.