Topic on Talk:Page Curation/Flow

(moved from Talk:Article_creation_workflow since this is a more appropriate place)

Hi, everyone. Thanks for all your ideas and feedback. I'm really excited about this project--I think we have a chance of making something really useful.

One of the things we've realized in talking with new page patrollers is that, in the absence of any real coherent UI, everyone has tailored their own custom interface from scripts, browser plugins, etc. This has its good and bad points, but it's also meant that we at the WMF can't actually examine the interface most of your are using, because we don't have it.

Any good software design effort starts with understanding what people are doing now. So, we're hoping that a few of you could send us screencasts of yourselves doing NPP. Here's specifically what we'd like you to do:

1. Begin recording a screencast of your NPP session. I think 10-15 minutes would be about right, but feel free to take more or less time as you see fit.
2. Clearly state your username, so we can easily link your comments with your video.
3. Start doing NPP. Spend about the first half of the time moving slowly, talking through the process and explaining what you're doing, why you chose that specific tool, what you like/dislike about it, and any other options you tried in the past. If you're using gestures or other off-screen interface elements, describe them. Also tell us about what you're looking for in the article, the policy decisions you're making, and your thought process. If you suspect copyvio, tell us what feature triggered that suspicion; if something seems like spam, what's spammy about it? Show us your process.
4. Spend the rest of the time running through NPP in your usual way. This gives us an idea of the pace at which things happen, how long specific steps take, etc. Don't try to go faster or slower than you normally would here. The idea is to understand where the bulk of the time and effort go, so we can streamline that process.
5. Wrap up the video.
6. If you're using the usertesting.com tool, upload the video per the instructions below. If you're using your own tool, send your video (the whole file, or a link if it's really big) to me at ian@wikimedia.org.
7. Send any additional comments you have in an email to ian@wikimedia.org.
8. Feel free to post a link to your video and your comments as a followup to this message. This is encouraged, but a video of one's personal computer is, well, personal, and I want to protect your privacy as well.

If you want to user the usertesting.org tools (this is the quickest/easiest route if you don't already have screencasting software), follow these handy instructions from Jorm:

1. Go to our usertesting.com launch page.
2. Click the "start" link (it's the only link available)
   • Do not close this tab or window. It's okay to background it, but don't close it.
3. The Java applet may require rights to run. Grant the rights as needed.
4. A few moments later, a "box" interface will appear. Resize it to contain your entire work area (either your entire desktop or all the tools you use at once)
5. Press the red button in the lower left of the interface to begin recording.
6. A countdown will start. Once the countdown has finished, the recording has begun.
7. Speak aloud your thoughts as you work. This is very important.
8. When you are finished, click the "Done" button (down near where the start button was)
9. The browser window (that you didn't close earlier) will then refresh with a screen cap from your session and ask you to upload the file.
10. Click "Upload this file". The upload will begin. This may take a while.
11. You'll be presented a list of screencasts. The filenames will be datestamps. Please note which one is yours and list it below, if possible.

If you have your own screencasting software, feel free to use it, and send your video and comments to ian@wikipedia.org. If you can't get the usertesting.com stuff to work (it's still quite beta) or you just want to get some screencasting software anyway, here's a list of tools, and here's another one.

Once we've received what seems like a representative sample, I'll compile all videos, we'll look them over, and we'll let you all know what we've learned.

Thanks in advance for your help. This will be invaluable information for making a tool that's actually useful to existing new page patrollers.Raindrift 20:59, 20 September 2011 (UTC)

There's also a great deal of material WikiProject Screencast, though it focuses on creating how-to screencasts for general consumption, not screencasts for user experience research. Do we want to make a MediaWiki-focused general documentation page for this here, including materials on usertesting.com?

Would closed captioning be an acceptable replacement to using a voice?

Absolutely.

This post was posted by 166.205.138.32, but signed as Raindrift.

In my very honest opinion, I believe requesting screencasts of new page patrolling is a demonstration of lack of WMF faith, recognition, and confidence in the stats based evidence that experienced users and admins have taken a year to research and produce through sheer dedication, determination, and long hours at the unappetising task of new page patrol. Add to that the WMF accusations that those researchers are a bunch of diehard deletionists. Adequate proof that NPP is a broken process manned mainly by a transient pool of raw beginners has been provided - now is the time to look for a solution that directly addresses the problem - nifty tools that streamline the work of those who already know what they are doing are most welcome, but they can be developed later.

Hey Kudpung: I'm sorry if you feel like the screencasting is about doubting your hard work on the quantitative statistics side. It's truly not. The request got put out because those working on engineering at the WMF, like Raindrift, are very interested in helping solve these problems, but they aren't New Page Patrollers. The point of asking for a screencast is to have you lend us a little of your hands-on experience in patrolling, which is what we lack. It's certainly not going to tell us the hard numbers which I think you're referring to, or refute anything of that kind.

Steve, I do understand, and I don't doubt for a moment that it is well intended. It may not be a demonstration of lack of confidence in the work that Snottywong, Blade, and I and others have already researched since October last year, but it certainly does appear that you are not satisfied with that feedback, and that this NPP Zoom project is still lacking focus on critical issues. I still feel therefore this request for screencasts will only serve to duplicate, rather than substantiate, the information that has already been made available to you and your team in the form of adequate tables and stats by mature and dedicated users - who are not , BTW, a bunch of dedicated exclusionists. Snottywong is an expert at providing data and has already provided plenty of scripts that were to be used to extract data for the ACTRIAL. If you require more tables and up-to-date data runs, I am sure that if he is asked nicely he would be only too pleased to oblige - he is after all, like me and Blade, deeply concerned with these NPP and new page creation issues.

As far as the request for screencasts is concerned, the wannabe Wikipedians who are patrolling new pages supercficially at the speed of light will not want to slow down in order to address such a request, while we, the experienced patrollers, are also content builders and admins, and won't necessarily have the time or inclination to engage on a mission so complex. It requires downloading special software - which I am not even aware of being available for my computer platfiorm - fiddling with it until it works, then uploading the files to some destination that you will no doubt create.

Because of the difference between the work of newbies (that will most probably never be made available as screencasts anyway) and screencasts of the more professional patrolling done by established editors and admins, I doubt that the screencasts will ever provide you with useful new information. I for example do most of my patrolling one one computer screen that has a dedicated Wikipedia window for NPP, while flitting back and forth from other Wikipedia tasks such as adding content, deleting pages, following up SPI, and answering new messages on my talk page.

Thus with all due respect, I would strongly suggest that for the time it would take for you to review the content of several three-hour screencasts, you may prefer to consider doing some new page patrolling yourself in that time, and gain some first-hand empirical experience. If other members of your WMF team, such as Raindrift (best if those who design cars have actually driven one), and perhaps those who heavily criticised our efforts at ACTRIAL without even reading the background to the trial, would also do the same, you would have some first-hand notes to compare.

You may also wish to review these slightly older, but equally important statistics prepared by BlanchardB: w:en:User:Blanchardb/CSD_statistics BlanchardB that provide a similar experience. The actual situation today is however far worse.

I've spent aa few hours over the weekend looking more closely into this screencast idea, and I'm beginning to see where it could indeed possibly help the developers understand more about the NPP process. Unfortunately, in spite of all my web design work, I've never needed to make any screencasts and I don't have the software to do it. None of Steve's suggestions work, and the only available solutions for my platform cost money. Perhaps the WMF could buy me a licence for one ;)

VLC Media Player can record stuff off the desktop (select File->Open Capture Device and select "desktop" for the capture mode), but you will need to edit the video afterward to add the audio track.

Unfortunately that didn't work either. FWIW I use Mac OS 10.6.8.

Hey, Kudpung. Thanks for taking the time to try recording a screencast for us. It'll be really helpful!

It just happens that I, also, run OSX 10.6.8. The usertesting.org screencast software works okay for me. It's rather beta, but I did manage to record a test with it. If it works for you, that's probably the most expedient way.

If you prefer to use a piece of desktop software, Telestream ScreenFlow is probably the best option. The demo is fully functional, but will leave a watermark on your exported video. That's fine, though. I'm sure we can ignore it.  :)

If you use ScreenFlow: It's not obvious now to stop recording if you haven't set a key combo in the preferences. If you switch back to the ScreenFlow application and hit 'File->New Recording', it'll ask if you want to stop the current one instead. When exporting, I recommend using the "Web - Low (multi-pass)", and scaling to the smallest size that maintains readability. 75% worked for me, but it'll depend on the size of your fonts. That produced ~500kbit video, which is small enough that 15 minutes of it could still be emailed.

As you might have surmised, we don't need the screencasts to understand the nature of the issue. The data you and others have collected are quite clear in that regard. Rather, we'd like to use them in designing a solution. I'm imagining a toolset that makes newbie page patrolling easier and more accurate, and provides a way for experienced patrollers to quality-check and mentor the newbies in a way that encourages them to learn, but also to disable patrolling for the ones who just can't get it. A whole host of very useful features depend on getting (mostly) everyone together in one place, which means we have to try to meet everyone's needs.

We're not terribly concerned with the workflow of newbie NPPers—we need to understand how they work to some degree, but it seems education is a more important goal here than efficiency. Also, I can become a newbie NPPer in short order, so that research is easy to do locally. Serious experience, on the other hand, is hard to get. It seems like each experienced NPPer has built their own UI to some degree, or at least uses Mediawiki very differently. To capture some of that functionality in the code we write, we need to see how you all work.

Hi Raindrift, thanks for all those tips. I'll try some of them out when I get home this evening. Please search around this confusing talk page - you'll find a thread somewhere I posted today with more detail on what I've got lined up for a screencast if I can get the software to work.

I'm still struggling with challenge of royalty-free screencast software, but in the meantime, i'd like to expand on one or two points you made. The serious NPPer probably hasn't built him/herself a sophisticated interface - they are not all geeks - they mainly just limit themselves to Twinkle - which is perfectly adequate, is quick, and takes up no screen space. The less experienced users often don't even know what Twinkle is - which ironically is probably not a bad thing, except that users do not get notified of CSD tags. I first found out what Twinkle is when (what seems) years ago, when I stumbled on a user page that had a "I patrol pages with Twinkle" userbox on it and I wondered what Twinkle was. I probably have one of the best possible working environments for optimal Wikipedia workflow, with several Mac Minis, a MacBookPro, all with large screens in a row and all sharing, and all operated from one keyboard and one magic track pad.

You say that you can become a newbie NPPer in short order, so that research is easy to do locally. Yes, and if you already have a solid knowledge of policy, page creation, and standard editing tools already, and spent two hours reading the instructions listed at WP:NPP, WP:DP, and WP:CSD and consigned most of the notability acronyms and around 30 CSD criteria to memoire, in less than an hour you'll be as proficient at NPP as our best patrollers. BUT, you are an adult, have an analytical mind, and probably a PhD - any page patrolling you do will not put you in the shoes of a 12 - 13 year old. However, you might wish to count how many pages you treat, doing all the steps required at WP:NPP, in one hour's solid patrolling without getting distracted, but you will be if you also have en.Wiki admin tools. What slows me down is that I do physical deletions too, ans sometimes salt pages and block the users when I'm patrolling. All this can dramatically reduce the actual number of pages patrolled. A 15 minute screencast may show me getting through 15 new pages or as few as three. There is also the fact that I cherry pick: from the front of the queue the page titles that I know are going to be a challenge for the average patroller, and of course pages with any spammy sounding names, and bios. At the back of the queue, I usually go for the bios first, then the ages with Indian sounding names. To save time, I go through about 10 - 20 picked pages on the list and let them start loading in separate browser tabs while I work. I have monitors and widows open with my 'tool box' user sub pages.

I should point out that I do a lot of work in the lesser namespaces, and quite often find material that is legally problematic and therefore needs to be oversighted. As such, I'm not sure it'd be a good idea to record my discovery thereof.

This... is a very good point.

Would it be possible for you to do your recordings in a namespace that is less likely to run into this type of stuff?

Perhaps you could share your page patrolling session in real time through screen sharing on a video conference with maybe Steve and Ian? I think the major issues are more about establishing how many pages you are able to fully examine during your patrol session, what percentage of new pages you empirically find to be really in need of rapid deletion, what pages are passed as 'OK' patrolled that should'nt be , and the frequency with which you have to correct a patroller's tagging. Also as an admin, roughly the percentage of new pages you summarily delete immediately.

Yeah, we'd be happy to do a screenshare any time.

I think we need to set up a screenshare session. I had another go with test.com. this morning. It took 29 minutes for a 10 minute session to upload, and when I got the result back after waiting another 14 minutes, it started and then just hung.

Okay, email sent about scheduling a time...

I've done a test run with Blade about screensharing andit worked perfectly. I'm nevertheless still looking into solutions for making screencasts on Mac. it looks as if the free trial of SnapzPro might be worth looking at.