Abstract/Background

Microcontributions and related user retention are essential goals for the Suggested Edits (SE) feed on Android. Depicts/image-tagging is an upcoming feature; we need to make sure users understand what this feature does and what it means to accept and reject machine-generated tag suggestions.

Research Questions

File page: Do participants expect more information about an image to add tags? Is the information that’s provided sufficient? Goal: find out if quick access to file page is needed.
Zoom: How would participants zoom an image? Find out which gesture to zoom images to incorporate: Tap to zoom, double tap to zoom, pinch to zoom.
Publish process: Do users realize that they can tap on multiple tags? Do users realize that their tags have been published?
New left/right arrows: Observe how users go back and forth between images that need tags. Do they understand the arrows?
Language and onboarding: Is the language that we’re using understood? Are users reading the onboarding dialog? Do they comment on it? Could be incorporated as a follow up question too.

Methodology

Usertesting.com
Test environments
- Mobile web on Android, using this test prototype link
- Alternate view of the same feature

Participants

Screener elements

primarily Android and Chrome browser user
Wikipedia user
Non-editor on mobile Wikipedia media

Demographic breakdown

3 US/UK/AUS/CAN
7 India and rest of the world
5 women, 5 men

Protocol

Screener

What is your primary mobile device?

   * 		iPhone [Reject]
   * 		Android phone [Accept]
   * 		Other [Reject]

Do you use Wikipedia?

   * 		Yes [Accept]
   * 		No [Reject]

Have you ever edited Wikipedia on your mobile device?

   * 		Yes [Reject]
   * 		No [Accept]

Tasks

Protocol

For the purposes of this test session:

imagine that your name is Sharvani and you’re encountering the following screens on the Wikipedia app on your Android phone
keep in mind that this is a prototype shown on mobile browser (some scrolling may be needed)

Take a minute or two to describe some of the things you can do on this screen.

If you haven’t already, take a closer look at the section ‘Image Tags’. What do you think this section represents, without tapping on it quite yet? 
Show us how you would begin adding an image tag. Comment on the screen you see next. 
Tap on ‘Get Started’ if you haven’t already. 
Now, go ahead and complete the task presented on this screen. Talk us through the process. Prior to tapping ‘Publish’, first explain a bit what you expect to happen or to see next. 
Tap ‘Publish’. What do you think of what you see next? Then, tap the continue button. 
And what are you seeing now? From this screen, tell us what you think would happen if you tapped on the left arrow next to the ‘Publish’ button. What if you tapped on the right arrow instead? No need to tap on anything, feel free to move to the next task. 
This time, take a second to review this screen. What do you think of it? How do you feel about the amount of information provided, about the image, anything else?  
Is there anything you would change or add/remove to make your experience on this screen better? 
If you didn’t mention this already, how do you feel about the images provided so far in this feature? 
Do you think you’d have a need to zoom in to the image or no? Either way, try zooming in. Did it work? If not, describe what gesture you would want to use to zoom into this image (e.g. tapping, double-tapping, pinching, or other). 
This is an alternative view of this same feature. How do you feel about this look? Let us know how you’d compare the two; which you prefer and why. 
Now, let’s go back to the first link (we left off on a screen with a photo of the Matterhorn). Show us what you would do next if you were on the Wikipedia app. Feel free to explore for another minute or so and voice any thoughts you have on what you see and experience.

Post-test Questionnaire

How did you feel about completing the tagging task? 
Briefly explain your understanding of the feature of adding image tags. 
What do you think adding image tags is for? How are your contributions through this feature used? 
Why might tagging images be helpful for Wikipedia users like you? Why might it be useful to do?

Findings

Element	Score	Finding
Language/Onboarding	✓	Participants did not have much commentary on the language/onboarding screen, but their understanding of the image tags microcontribution task was generally quite clear. A few participants experienced initial uncertainty based on the initial description and onboarding screen, but upon seeing the task screen itself, quickly grasped both the task and the related task outcomes.
Publish process	✓ ✓	All 10 participants understood that they could select more than one tag and the tag published confirmation was clear. Confirmation screen 6 of 10 expected to see the photo with the selected tags or an ‘are you sure’ screen in addition to a confirmation message. However, none were deeply offended and a couple mentioned the status quo offered a minimalist interface and may mean quicker processing.
Left/Right arrows	✓	6 of 10 participants described the correct expectations for the left and right arrows next to the Publish button. The remaining 4 more or less had the same expectations, but some distortions in understanding include answers like ‘I’m not sure’ or thinking that the arrows would lead to more ‘options for tags’ in addition to navigating amongst image options.
Information provided	✓	8 of 10 participants indicated they felt the information provided on the tagging screen is satisfactory/sufficient. Participant concerns: Why are there so few tags (1) Layout not good to look at (1)
Images provided	✓	7 of 10 were positive on the images they saw using this prototype. Negative commentary includes: Poor layout/overlay look ‘dark/dull’ image ‘a bit blurry’
Image Zoom	-	All 10 participants defaulted to the ‘pinch’ to zoom/interact with the images. Most (6) mentioned they didn’t feel a need to zoom was necessary.
Prototype Multi-tag select view or Static Single-tag rate view	-	Prototype (5) Static (3 - researcher note: would discount these votes. participants had real issues of understanding with the static view, like missing the ‘bridge’ tag altogether, and not understanding the ‘vote’ action. Additionally, 2 votes were for vague aesthetic reason; my hunch is because the overlay did not slightly cover the bridge image on the static page.) Tech difficulties, no answer (2)
What did participants do/want to do next?	-	Publish another set of tags (6) Search for image to confirm publish (1) See all images to which participant contributed tags (1) Explore other tasks on SE screen (1)
Suggested Edits main	-	3 of 10 participants described the SE main screen as if it were the ‘administrative’ center for all app editing/management of editing capabilities.

Recommendations

Element	Recommendation
Participant suggestions	Evaluate and address the following as is needed/valuable/worthwhile: Add own tags (7 participants) Add own photos (1) Search other existing tags to add (1) Add photo credits (1) Add button to bookmark image link (1) Add day/night view (1 - researcher note: uncertain how this look will work with the light/dark views on Android. Will this override user settings or vice versa?)
Publish process	Consider the utility of including a link to the image or a visual confirmation for users to see their added tags against the quick/clean current.
Image zoom	Allow for users to zoom (if it's not much trouble, so they can do so if they'd like) on the image with the pinching gesture (most intuitive)
Prototype Multi-tag select view or Static Single-tag rate view	Stick to the former for clarity and efficiency’s sakes. The static version in practice will not necessarily always look ‘cleaner’ and it appears to create more confusion than expected.
Suggested Edits main	Consider the macro-level goals around user participation and type/extent of editing on Android app with regard to the evolution of the SE main screen. Some participants’ impressions of it as an app editing administrative hub could be a positive, a diversion, or a combination of both.