Account creation user experience/User testing

We asked average Web users to complete account creation on Wikipedia (as well as a test wiki) in order to gather qualitative feedback about the experience and pinpoint any problems we may have been ignorant of in our A/B testing.

Background

We gave users the following instructions.

In this scenario, you will be testing the registration page for Wikipedia. We are trying to improve the experience for them, so we'd like to know what the experience is like. Remember, we are testing the interface, not you. If you are having difficulty with something, the problem is with our design. Please "think out loud" as much as possible; tell us your thought process during each task, and try to explain your general opinions as you arrive at them.

Go to https://en.wikipedia.org/

Create an account, once you have found the registration page. You may choose any user name, password, and other credentials you prefer. Be sure to talk through your decision-making process, and explain what you find helpful or frustrating.

Once you have successfully created an account, please answer the questions...
Did you have a Wikipedia account prior to this test?

What, if anything, frustrated you most about the registration process?

What did you like about the registration page?

Standard experience

Tested with two users ACUX_control on English Wikipedia. The second of these tests included the only user who had registered and edited in the past.

The Good:

All users were able to quickly find the “create account” button.
All users were able to create an account.

In both forms, several people noticed and appreciated that email was optional.

The Bad & The Ugly:

Presence of username policy and other content definitely slowed down users. Even when they said they would ignore it, they had to scroll past it, and it increased the length of the tests. Choice quote: “A lot of stuff to ignore.”
One of the testers thought it was strange that there was an asterik next to the option email field, since usually required fields are marked that way.
Only one user got validation errors, but was clearly annoyed by it. Choice quote: “This could be a pain.”
Users did actually read the landing page post-registration, and noticed the call-to-action regarding preferences. One woman spent the majority of her 8:41 test changing her preferences, tab by tab. This was very painful to watch.

Test 1 of the current account creation experience, being used as a control in our A/B tests
Test 2 of the current account creation experience

ACUX

Tested with four users on ACUX_3 on Piramido.

The Good:

All users were able to create an account.
It was very fast to get through ACUX_3. The minimum time to create an account was 1:13. This may be helped by doing it on a test wiki where there are no username collisions, but still good news. Choice quote: “Much quicker than I expected.”
Users clearly read the instructional text in the fields.
Users noticed and appreciated client-side validation and the ui for confirming their choices.

The Bad & The Ugly:

The form used default field border styling, which looked inconsistent depending on the operating system. This will be fixed in Agora Login.
All of the users got the error about username being blank before entering text. Thankfully though, they ignored this error.
The CAPTCHA was the biggest hurdle to users in ACUX_3. They hated it. Choice quote: “This is ridiculous. I can’t even see this.” “I feel like it’s a little hard to read.”
When a user was stalled by the CAPTCHA, she read the “we can create an account for you” and was clearly confused by it.
In both forms, the fact that field content isn’t remembered on refresh frustrated users. On ACUX_3: “That’s irritating. Now I gotta go all the way back and do it again.” A second user didn’t even see that she had to go back. Could we consider adding something like garlic.js ?
One user was able to skim the benefits, but had to read the "Community" benefit more closely. Likely because the headline was not descriptive enough - it may need to be changed.
One user was hovering over benefit headlines, likely assuming they were links. These possibly need to be restyled.

Test 1 of the new account creation
Test 2 of the new account creation