Translation UX/Design feedback 1

Scope: First round of feedback collection and testing for Translation UX/Wireframes A and Translation UX/Wireframes B

Create simple interactive prototypes for the wireframes
Compare design options by asking users to perform simple tasks
Gather user/team feedback and requests for improvement

User test scenarios

Test 1: New translator registration process

Goal: A potential translator can register on TWN and contribute a translation in his language

Scenario: You are a reader of wikipedia in your language. Seeing a community discussion on translations and translatewiki on a talk page, you decide to find out how you can contribute translations in your language. You open translatewiki.net in your browser

User instructions:

Understand what the TWN platform is and how you can contribute to it
Register yourself as a translator for your language and the projects that you are interested in
The activation mail takes you to the account confirmation page on TWN, proceed to contribute your first translation.

Observations:

Does the user understand the purpose of TWN?
Discoverability of the registration process
Fluency of the process. Does the user has doubts on how to proceed?
Transition to translation. Is it clear how to continue after registration?

Test 2: Translation search

Goal: An unregistered translator can find and fix an incorrect translation

Scenario: You are an active editor of wikipedia in your language. You one day notice that the link for 'Random Article' has a typo in your language and you wish to correct it. You decide to make the correction using TWN, but have never used it before. You open translatewiki.net in your browser

User instructions:

Search for the incorrect message you are trying to fix
Use the search options to make sure you have found the correct message and project
Use the available tools to submit a corrected translation for the message

Observations:

Test 3: Translation workflow

Goal: A translator can translate and review multiple translations with minimum interactions

Scenario: You are an existing translator of TWN in your language. You like to maintain 100% translations of mediawiki extensions in your language and also the occaisional proofreading. You want to try out the improved translation workflow that is in beta testing. After logging in to TWN, you see the new translation interface.

User instructions:

Use the project options to load untranslated messages for mediawiki extensions

5 messages are loaded:

with suggestion http://translatewiki.net/w/i.php?title=MediaWiki:Antispoof-empty/en-gb&action=edit&loadgroup=ext-0-all&loadtask=untranslated
with suggestion, variables http://translatewiki.net/w/i.php?title=MediaWiki:Antispoof-blacklisted/en-gb&action=edit&loadgroup=ext-0-all&loadtask=untranslated
no suggestions, variables http://translatewiki.net/w/i.php?title=MediaWiki:Antispoof-conflict-top/en-gb&action=edit&loadgroup=ext-0-all&loadtask=untranslated
outdated/fuzzy http://translatewiki.net/w/i.php?title=MediaWiki:Antispoof-noletters/en-gb&action=edit&loadgroup=ext-0-all&loadtask=untranslated
long message, incorrect translation, pending review http://translatewiki.net/w/i.php?title=MediaWiki:Centralauth-merge-welcome/en-gb&action=edit&loadgroup=ext-0-all&loadtask=untranslated
pending review http://translatewiki.net/w/i.php?title=MediaWiki:Antispoof-conflict-bottom/en-gb&action=edit&loadgroup=ext-0-all&loadtask=untranslated

Translate #1 by using suggestion
Translate #2 by typing, insert variables manually
#3 is unclear and without documentation. Ask for clarification
#4 is correct, flag as reviewed
#5 is spam, flag as incorrect/delete
#6 is correct, flag as reviewed

Observations:

Are aids understood and useful when needed?
As about perception according their previous translation experiences (with TWN or other platforms).

Personas

Peter, a wikipedia contributor from Canada

Goal: Correct typos in the UI so others had a better experience.
Languages: English + French
Device: Mouse + Keyboard

Scenarios

Finds a message in the UI which had a bad translation in French and looks for it to improve it.
Finds a message in the UI which had no translation in French available and looks for it to improve it.

Carla technology enthusiast from Switzerland

Goal: Translation as a way to contribute to a software project.
Languages: English + Italian + German
Devices: Keyboard mainly.

Scenarios

Find untranslated messages in Italian and German for her project and translate them.
Find outdated messages for her project and verify if they are ok.

Lev, a professor from Israel

Goal: Help his language community.
Languages: English + Hebrew + Russian + Old Aramaic
Devices: Mouse + keyboard

Scenarios

Find messages to translate across projects to provide transations.
Review translated messages in Hebrew to ensure they are properly translated.

Behavioral variables

Number of languages the user can translate (English + N)
- Two
- Three
- More than three
Kind of message
- Length of the message.
- Context info provided.
Input mechanism
- Keyboard-only
- Mouse + keyboard
Motivation:
- Contributing to a project
- Help your language community by providing localize software
- Correcting a specific message
- Practice a language
Translation expertise:
- No experience (new)
- Little previous experience
- Usual translator

Prototypes tested

Sign-up prototype: v1 v2
Translate-workflow prototype: http://dl.dropbox.com/u/30377416/prototypes/translate-ux/translate-workflow/ru/index.html
Search prototypes could not be prepared in time due to complications in translating it for test participants

Test Observations

Pilot Test

The user could complete the test correctly.
Top breadcrumbs and message filters were perceived as useful.
Bug: Top language indicator says Spanish regardless of the current langue.
Surprised initially that there was not “save and go next” (he was used to previous UI)
but when he realized it was the “save” button, he used it fluently.
Found strange that “hide description” does not completely hide the description.
Commented on the lack of the ability of deleting translations for the current UI.
When a translation is marked for review it was confusing to see that "confirm" button becomes active
We could try inplace undo, and activate button only on edit.
Did not noticed the expand function
Thinks translations messages should be short and emphasizing this is more important than provide support for long messages.
Did not noticed the inter-language translation access.
But when used, found it difficult to know in which language the translation was supposed to be.
A clear label may be needed.
The SPAM example is not representative, it is probably better an incomplete translation.
When there is spam in the current system, current options are: translate correctly, admins can delete, inform in support.
In some cases, message keys are needed. Two main purposes: find a specific key (search can be used), find messages that follow a similar structure.

Test with user #11 on 17 Aug 2012

Languages: Speaks Macedonian, Russian, and Serbo-croatian, but only provides translations to Macedonian (his native language) since "he is a perfectionist".
Location: New Zealand
Technical issues: His internet connection was really bad and we could not use screen sharing.

Sign-up test:

The usser guessed that Maori was provided as an option because he was located in New Zaeland.
Did not indicated his language.
We need to make it clear that adding at least one language is mandatory.
At first the user does did not understand that he was supposed to translate a message
He was not able to identify source message and associate the "suggestion" label with the suggestions provided.
When given a seconthought, he was able to complete this step.
The explanation for the message is read after the translation is provided (when it is probably not useful)

Translation workflow test:

The user noticed it was a big change from the current UI (to which he commented he was used), but he commented that this one looked "fancy" and "good".
The labeling system for messages (translated, outdated, to review) was found clear
The user commented that "outdated" was much better than "fuzzy".
The user identified the main purposes of the "requires revision feature" ("When you are not completely sure of how to translate" or "When there is a rule that must be checked by somebody else")
According to this, we can even consider a secondary "send to" feature.
The user did not noticed the ">>" expand icon
When used, he compared it to opening the message in a new window with the current UI.

ULS test: we asked the user to change among different languages in TWN, but connection was so bad we could not communicate. He will send comments by mail.

Test with user #9 on 20 Aug 2012

Started translating 1.5 years ago, initially motivated by a translation rally.
"If there is strange wikitext, I just skip it because I don't want to break anything".
Languages: Speaks French and English.
Location: France

Sign-up test:

The user marks French as it was one of the suggested languages.
The user liked the use of a translation to confirm users.
The user commented: "I like the fact that I am useful from the begginning".
The use of arrows and circles to present the suggestions was confusing to the user (he associated it with some kind of worklow to follow).
It was not clear the purpose of the "quick menu".
The user understood the translation progress bar for his language. He said he views project statistics regularly and it motivates him to continue translation.

Translation workflow test:

Compared to the current UI, the user found the prototype "more useful", "easier to work with" and "requiring a small ammount of space" (compact).
The user found not much useful the "in other languages" information (it was provided in Italian in th eprototype which was not oneof the user languages).
The user noticed that shortcut desciption text was not clear
It should be explicitly stated that skip does not save changes.
User tries to skip message 2 since he does not understand completely the use of parameter $1.
The "requires review" function was not understood initially.
Once used, the user commented "it helped to be more motivated" because he "was always scared to do something wrong".
The user did not noticed the ">>" expand icon and function was not clear to the user.

ULS test:

The user found English by looking at the list of languages as the first attempt.
The user used search for "Greek" and then the map as a second option but he could not select the language since did not know the Greek authonym.
The user used search to locate French.

Summary

Feedback

The reaction to the new designs from test participants have been overtly positive. Participants commented that the redesigns are an improvement over the current interface and they had no issues adjusting to a new translation interface.

Issues

The process of scheduling tests can be improved by asking potential testers favored time and day of the week beforehand for greater chance of the participant accepting.
There is no simple way to translate prototypes for participants. The current workflow is oneway, making prototype changes after translating difficult. Alternative tools and workflow need to be explored