Wikimedia Research/Guerilla testing Wikigrok

Guerrilla testing Wikigrok prototype, iOS mobile (phone)

In order to get some quick feedback on the initial idea of getting people to fill in Wikidata information via the Wikipedia article on the item, we built a lightweight prototype and handed it to people to get their reaction.

The goal of this research was to observe people:

Reading an article on the mobile site (on an iPhone)
Their reaction to the popup – whether they would notice it at all and whether they would interact with it
Whether they understood the question and were able to answer it
Where they thought their contribution went, what happened to it
Overall, if they enjoyed the experience

To get started, we handed people an iPhone with the prototype loaded up and asked if they'd heard of Paul Rand. When they said no, we asked them to look at his page on Wikipedia and skim through it as they normally would to find a bit of information about him.

Where

The Barbican Center in London during Wikimania 2014

Findings: Patterns observed

3/5 people (interestingly, all of the women...) thought it was a quiz and didn't like that aspect of the experience.
Seeing the x to dismiss made people more comfortable with the experience (though they got very frustrated when it didn't work like expected).
Though some people seemed to have a negative reaction to the feature, everyone was able to answer the question after a very cursory skim through the lead section of the article.
No one understood where the contribution went, though this didn't prevent most people from continuing to answer more questions once they did one.

Bugs and/or Suggestions

1st question should be whether the person wants to help Wikipedia/improve Wikipedia information, etc. – then ask the real questions
Only thank user after 1st answer, not subsequent
A secondary info page that describes why this contribution is important

Raw Notes

Maryana's notes

20s/30s woman, never edited Wikipedia

"it's a test!" – sounds disappointed, doesn't like it
"it doesn't go away" – taps to dismiss but it doesn't go away, which frustrates her
is able to answer the question despite not liking the experience
thinks answers go to a "statistical database," which she doesn't like b/c she thinks it records her data and "feel like spying"
doesn't edit but loves Wikipedia

30s woman, edits Wikipedia occasionally

"it's a quiz!" – like the other woman, doesn't seem very happy to be "quizzed/tested"
"kind of distracting if you're just looking something up" but okay "if I can just close it"
thinks it's used for "article assessment"

20s man, never edited Wikipedia

thinks it's used for "verification"
"of course I want to help Wikipedia" – but that's because he's a Wikimania attendee & loves Wikipedia :)
wants it to check whether he's right or wrong
doesn't find it too intrusive – "I can just close it"

Sherah's notes

20s man, never edited Wikipedia

"it keeps popping up and that's annoying"
"the graphics look like an ad"
"I will always say no to participating" – in answer to question if the first question was whether he wants to help and second question was actual Wikidata question
"I don't know if my answer was right or not"
"it's too much interaction"
"I don't know the info being asked. I'm here on the article to learn, too!" – but he's able to answer the question after just skimming the article

20s/30s woman, never edited Wikipedia

Not a smartphone user – have to show her how to scroll
"It's a quiz. It's just a quiz. I have no idea." Not happy.
"If I'm looking for something, I don't really want to answer something. It's distracting."