Editing Tasks Survey

Group:	Design Research
Start:	2015-05-15
End:	2015-08
Backlog:	#WMF-Design-Research
Lead:	Aripstra (WMF)

Goals for Survey

In order to inform the design of editing tools for wiki projects, the design research team, in collaboration with the Editing team, is creating a survey to better understand these questions and gather some informative data:

What are the most frequently done editing tasks for "casual editors," "active editors," and "power editors" (defined below) in both the wikitext editor and Visual Editor?
Of all the tasks we could think to list, how easy or difficult are they for casual, active, and power editors to perform?
What patterns do we see in qualitative feedback that casual, active, and power editors can provide?

How are we going to get the answers?

For a survey like this to provide useful design information, we need to make sure to invite the right people to participate. For example, if we asked people who only read Wikipedia to participate in this survey, we would not get accurate answers, as readers do not have any editing experience. So, after a series of conversations about how to invite the right participants, we have decided to do the following to invite participants to take this survey:

Define the two segments of users to invite to participate:
Design and build the survey in Qualtrics and pilot with a small group of users.
Work with Analytics to query the database for possible participants.
Send the survey out to all possible participants.
Analysis on the collected data.
Report out the findings here on this page.

Choosing the right survey participants

Three editor groups were identified to be contacted for the survey based on the monthly frequency of their edits in August, September, and October 2014.

Editors who had less than 5 edits per month in each of the three months. ("casual editors" in each month)
Editors who had between 5 and 100 edits per month in each of the three months ("active editors" in each month)
Editors who had more than 100 edits per month ("power editors" in each month)

Note that since the frequency of edit counts changes from month to month for each editor, and an editor who is active in one month can become a power editor in the next month, we assigned editors to each of the above categories based on whether or not they consistently stayed in the same category in the most recent three months (August, September, and October 2014). For example, we assigned an editor who has edited 100+ edits in the three months in group 3, and we did not consider an editor who has done 100+ edits in August and October, but not in September, in any of the three groups.

Based on the above criterion, we queried our internal data-bases for English Wikipedia users and found 9316 users to contact.

Findings

This is a summary of the findings from this survey. Please dive into the linked presentation to dive into details and reported quotes.

The "basic editing tasks" that were reported in this survey (by new, casual and experienced editors) are:

1. Add / edit text (small typos and grammar)

2. Add / edit text (sentences and paragraphs)

3. Add a wikilink

4. Add / edit a citation

5. Format text

Most frequent other tasks (besides basic editing) that people do:

Reverting vandalism (new 1 / casual 16 / experienced 23)
Admin actions (casual 2 / exp 10)
Advanced image action (casual 3 / exp 4)
Template-related (casual 2 / exp 5)
Tagging/categories (casual 2 / exp 4)

Some feedback on the wikitext editor

Tables are really hard
Experienced editors prefer it as they are familiar with it and have gone up the learning curve to figure it out.
Some experienced editors think everyone should do the same learning they had to do, other editors think editing should be easier, and VE is easier for new editors.
It is not intuitive for new users.
It works well and is consistent for those who know how to use it.
Some casual and experienced editors are afraid WTE will be phased out.
A lot of requests specifically for a syntax highlighter and also other gadgets and features.
Requests for better help and tutorials.

Some feedback on the visual editor

Casual editors say there are things they can’t do in Some casual editors say there are things they can’t do in VE that they can do in WTE.
Some trust issues - (not knowing what VE is actually doing what it looks like it is doing)
Some (30) experienced editors say they will never use VE, they prefer WTE.
Some experienced editors say they will never use it, but it might be useful for others
A few experienced editors think VE is confusing and complicated.
A few experienced and casual editors think VE will be good
Some hate of VE was expressed as expected. (20 casual and 30 experienced)
Casual editors shared some compliments
4 experienced editors said they think the VE project has been poorly managed
A few feature requests (many of which have been done