Topic on Talk:Edit Review Improvements/New filters for edit review

Collaboration team is announcing plans to graduate the New Filters for Edit Review out of beta on Watchlist by late June or early July. After launch, this suite of improved edit-search tools will be standard on all wikis. Individuals who prefer the existing Watchlist interface will be able to opt out by means of a new preference.

The New Filters introduce an easier yet more powerful filtering interface to Watchlist as well as a whole list of filters and other tools that make reviewing edits more efficient, including live page updating, user-defined highlighting, the ability to create special-purpose filter sets and save them for re-use and (on wikis with ORES enabled) predictive filters powered by machine learning. If you’re not familiar with the New Filters, please give them a try on Watchlist by activating the New Filters beta feature. In particular, it would be very helpful if you can test the new functionality with your local gadgets and configurations. The documentation pages provide guidance on how to use the many new tools you’ll discover.

Over 70,000 people have activated the New Filters beta, which has been in testing on Watchlist for more than eight months. We feel confident that the features are stable and effective, but if you have thoughts about these tools or the beta graduation, please let us now with a post on this talk page. In particular, tell us if you know of a special incompatibility or other issue that makes the New Filters problematic on your wiki. We’ll examine the blocker and may delay release on your wiki until the issue can be addressed.

This is neither here nor there, but on an old machine (2009 macbook, 8gigs ram, SSD) the filters take a very long time to load on a large watchlist. I really enjoy them but use the old design rather than wait for the page to load. I'll be opting out, but look forward to using them with a new machine!

Hi @Amorymeltzer. Thanks for the kind words about the filters, but I'm sorry to hear load time is such a problem for you. Frankly, a 2009 Macbook isn't all that old. We'd like to look into this a bit more. Can you tell me a) how many pages are on your Watchlist, b) how many items you choose to show at a time, c) what browser you use?

If you have time, it would be especially helpful if you could turn on the New Filters beta and put in the settings you use, then copy and paste the URL (which contains all the settings) for us, so we can see what you're trying to do. i

I'm copying @Catrope and @Etonkovidova, since they will be interested in this thread.

@JMatazzoni (WMF): Sure. For starters, I use Modern with a dumb amount of userjs, including a number of scripts that dramatically slow down the watchlist (e.g., user highlighting, shading out already-seen pages, cross-out blocked users). In the interest of this, I've tested it using vector, for which I have no customizations or js loaded.

I'm on OSX 10.11.6, running Firefox 60.0.02. My watchlist just hit exactly 5,500 pages (although I had the same issue when I had it pruned to around 3, 3.5k), and I show 1,000 changes (that's ostensibly over the last 7 days, but assuming there's minimal drama at enWiki (lol), it usually spans about 24 hours).

Without the filters, it took about 13 seconds to load, whereas with the filters it took 27 seconds. Nearly all the extra time was with the loading dots for the filters. That url is: https://en.wikipedia.org/wiki/Special:Watchlist?hidebots=1&hidecategorization=1&hideWikibase=1&limit=1000&days=7&enhanced=1&urlversion=2 (no bots, page edits, page creations, logged actions). From there, adding a paramter (https://en.wikipedia.org/wiki/Special:Watchlist?damaging=verylikelybad&hidebots=1&hidecategorization=1&hideWikibase=1&limit=1000&days=7&enhanced=1&urlversion=2 aka very likely have problems) takes about 12 seconds to show. The interface itself is fairly laggy, often requiring a second or two after clicking an option before something registers.

FWIW, with modern and my userjs, it was 17 with the old system and 30 with the filters. Happy to provide more info if you like; I know I'm an edge case, so I appreciate the thought regardless.

Hi @Amorymeltzer. Thanks for all the detail. The engineers request that you try adding the following to your URL and see if it changes the times: ?safemode=1

That will turn off gadgets and user scripts, which will be a big help in diagnosing where the delay might be coming in. Let us know what you find!

@JMatazzoni (WMF) Sure, safemode is no issue. Below is average of two trials for each skin.

• Vector: 18s with beta filters, 8s on the old system
• Modern: 20s with beta filters, 9s on the old system

As before, majority of extra time is waiting for the panel to load, so technically the watchlist itself is visible.

@Amorymeltzer As you might have figured out, 12 seconds is the amount of time it takes to query the database and figure out what the recent edits to pages on your watchlist are and which of them match the filters you chose. This is a known problem with large watchlists, unfortunately, and affects both the old and the new UI. You also get hit by that 12-second delay every time you change your filters, because it has to rerun that query. There may be some optimization we can do for large watchlist, I'll look into that later if I have time.

As for the additional delay caused by the new filters UI, that should only depend on the number of results. I tried with 1000 results on RC (my watchlist isn't that big) in Firefox, and it only spent 0.6 seconds in the loading dots stage. I suspect gadgets and/or user scripts may be interfering here, which is why Joe suggested using safe mode, which disables those. If it's still slow with safe mode, we can do a more detailed performance analysis of your particular case; if you're interested in that, get in touch with me on IRC (RoanKattouw) or email/gchat (roan at wikimedia dot org) and I'll walk you through it.

Whoops, our messages crossed. It looks like it's still quite slow for you in safe mode, which I'm very surprised by. If you're willing to spend some time investigating this issue with me 1-on-1, I would be quite interested in that; it's hard to impossible for us to fix issues that we can't reproduce or observe ourselves.

Yeah, I figured as much — I just tried dropping down to 100 from 1000, and the whole page with new filters loaded in about 9 seconds with safemode=true. At any rate, I know I'm an edge case here, so I do appreciate the thought. The filters really are excellent. And yes, anything that can every be done to speed up large watchlists will always be appreciated!

Just to add a little observation I had after seeing this interesting discussion. With my very tiny Watchlist consisting of only 29 pages and absolutely no user scripts that customize what appears in Watchlist, it took me 2 seconds for the list to appear and around 3 more seconds for the panel itself to load 500 changes. This is on a Firefox Nightly 62.0a1 on a Dell Inspiron 3542 running Debian 9. fast.com reports my internet speed as 4.8 Mbps which I find to be a decent speed in the place I live.

I'm reporting this as I find this very slow as compared to the 0.6s observation by Roan Kattouw (WMF).

I forgot to mention that this was for the URL,

https://en.wikipedia.org/wiki/Special:Watchlist?hidebots=1&hidecategorization=1&hideWikibase=1&limit=500&days=7&urlversion=2&safemode=1

I'm also seeing severe performance issues. The old watchlist takes 8 seconds to fully load, and is usable after ~4 seconds (the remaining 4 seconds are for the ORES highlighting to load). The new watchlist takes 34 seconds to load. While the watchlist comes on screen after 4 seconds, I cannot interact with it in any way until the three grey dots go away 30 seconds later.

I'm using the corporate-mandated IE11 running on a fairly modern computer, albeit inside a virtual machine. However, performance on other websites isn't this bad.

If the issue is large watchlists (mine has ~9000 pages), perhaps the new system should fall back to the old one once the watchlist is above a certain size. However, despite what @Roan Kattouw (WMF) said, my huge watchlist doesn't take 12 seconds to load in the old watchlist page despite my list being larger. I suspect that this comes down to server-side processing being faster than whatever the new watchlist is doing.

Thanks for your report @Ahecht. Can you please copy and paste the url when you're using your default filter set, so we can look for any patterns with that? Meanwhile, I'm pinging @KHarlan (WMF), who's also looking into this issue.

@JMatazzoni (WMF) https://en.wikipedia.org/wiki/Special:Watchlist?hidemyself=1&hidebots=1&hidecategorization=1&hideWikibase=1&limit=500&days=7&enhanced=1&damaging__verylikelybad_color=c5&urlversion=2

@Ahecht thank you for the URL. A few questions:

1. Can you please try appending &safemode=1 to the URL, starting IE11 in "No add-ons mode" by running the Run command from the Start menu, and then typing iexplore.exe -extoff into the box, and then visiting your Watchlist page again?
2. Could you please elaborate on "I cannot interact with it in any way until the three grey dots go away" -- you cannot click on links in the watchlist? Are there other things that are blocked, scrolling for example?
3. Do you also encounter performance problems on Special:RecentChanges?

thank you for your help!

@KHarlan (WMF):

1. After my last post, I took the opportunity to trip my watchlist down to 6750 items, which brough the load time to 20s. &safemode=1 shaved off about 1.5 seconds, and starting IE in no add-ons mode didn't have a measurable impact.
2. Yes, I cannot click any links or scroll. I should mention that the three dots also stop animating about halfway through.
3. Looks like I have the same issue with RecentChanges taking about 20-30 seconds to load. Hadn't noticed that before, as most of my recent changes patrolling is using Huggle.

@JMatazzoni (WMF) With 250 changes, it brings is down to a little over 10 seconds (with safemode)

Thanks @Ahecht, could you also please let us know the load time with a limit of 100 items, and also if you find the performance with 100 items (page load, toggling filters, etc) to be acceptable?

@KHarlan (WMF)That brings it to 6 seconds. While 6 second by itself is acceptable, the fact that I'd have to load the page 10 times (for a total of 1 minute) to see the same number of pages that I get on the old watchlist (which loads in 6 seconds now that I've trimmed down my watchlist) isn't.

@Ahecht I hear you :) thank you for this information, we will look into it further.

@Ahecht, the team is looking into this, so please answer another question. What happens if you reduce to 250 changes (instead of 500). As a side note, if you do end up reducing the search in this way, it might work for you to try using the "Unseen" filter (in the Watchlist Activity group). With that filter on, once you've looked at the first batch, you can move on to the next. But anyway, we're just trying to figure out how the system is working, and none of us has a test account as large as yours!

> none of us has a test account as large as yours!

Maybe you could ask if @Ahecht is interested in sharing their watchlist?

en:User:Ahecht/watchlist

Thanks! I guess it's now upto the people involved to use it to test and fix the issue.

@Ahecht @Kaartic @Amorymeltzer thanks again for your feedback and help with diagnosing the performance issues. We're working on a few fixes, you can track our progress at https://phabricator.wikimedia.org/T197168

@Ahecht @Kaartic @Amorymeltzer we've deployed a set of fixes to address performance issues, if you have time we'd appreciate your feedback. Please look at "Send us your performance traces" in https://phabricator.wikimedia.org/T197168 for instructions on sending us your feedback. We are still investigating a few more optimizations as well but hope that you will see an improvement with the code that's been deployed already.

Performance is much better. With almost 9000 pages on my watchlist, viewing 1000 pages at a time, the page loads in ~9 seconds. The watchlist appears after 6 seconds, and the three dots take an additional 3 seconds to go away. I posted more details on Phabricator.

Thanks for your report @Ahecht! I just rolled out some more improvements, so hopefully it'll be a bit faster still if you try again.

Said as much on the phab, but the improvements are fantastic! Barely any difference now between old and new. Thanks so much!

Very happy to hear it, thanks for the feedback and help with the performance traces!

I also could confirm that Special:Watchlist loads faster than I observed previously for my tiny watchlist.

Thanks for these reports! We're now satisfied that the performance problems are under control, and a few days ago, we scheduled the deployment of this feature. The deployment date is July 9th for most wikis (mostly smaller ones) and July 16th for the remaining wikis (mostly larger ones). This was announced on Phabricator and in Tech News, but I forgot to announce it here.