Translation UX/Development plan

This is the development plan for the translation user interface project. It is a continuation of the design documents and presentations, the knowledge of which is assumed for the reader.

This document is used by the Wikimedia Language Engingeering team to organize development work and dependencies. The development is slated to start in late 2012. For the current status, please see the team's mingle.

This document was prepared by Niklas Laxström and it has been reviewed by the Language Engineering team.

Introduction

After the Universal Language Selector, Translation UX is the second major development project of the Wikimedia Language Engineering team which involves design and user testing efforts. Pau Giner and Arun Ganesh have come up with new designs for three parts of the translator's workflow: signup, translation and search. In this document each of these three parts are called a track. Each track will be described in its section below.

The three tracks are to some extent independent, but there are dependencies between them. For example the simplified translation editor for new translators should not be deployed before we have the full version of the editor ready. Also, removing the display of message keys from the message listings might affect some uses cases, which could be covered by the search track. Given these dependencies, it makes sense to start with the translation track and start work the other tracks alongside, when there is time and opportunity to work on them.

For each of those tracks we need to develop new features and adapt existing code. The search, signup and translation workflows already exist, and thousands of people are already using them.

In principle each track could be developed in parallel. Each milestone should be able to fit in a two-week sprint, but not necessarily take the full team's capacity for a sprint.

The Translate UX development does not lack challenges and they are mostly different from those in the Universal Language Selector project. One big issue is to get clarity on how we can and want to do tracking to evaluate the changes.

Challenges

Existing codebase. Translate, where the developments of this project will concentrate, is a mature extension. It's been developed for many years and contains many lines of code, mostly in PHP. Unlike in the ULS, we are not starting from scratch and there are many limitations set by the existing design, as well as by MediaWiki. Some of them we can fix or work around, some of them not. This project will stress-test the internal interfaces.
Release early, release often. The Translate extension code is used by the Wikimedia Foundation and translatewiki.net. Every two weeks a new snapshot of Translate is made, to be then deployed during the following two weeks. The language support extension bundle will take snapshots every month once it is started. We have to be extra careful and ensure nothing breaks while we refactor and add new code. This makes it important to have good test coverage: PHPUnit for PHP, QUnit for JavaScript and possibly Selenium for workflows. The team has only one QA person, so the amount of manual testing that can be done is very limited. Whatever can be tested automatically should be set up once, and repeated automatically as often as needed.
Existing userbase. Translate has many existing users. While the feedback in user testing has been positive, we need to maintain old and new feature sets simultaneously, temporarily complicating the code base. Only once features are solid enough to be launched standalone, and further testing and analytics confirm that they fare better, we can drop support for old code. Using dark launches or A/B testing is necessary to gather feedback during development. A gradual deployment can consist on several of these stages: eat our own dog food (internal use), ask for invitation (feature is announced and users can request to test it getting immediate access), bucket (some users get the new, the whole rest gets the old, while we compare), show the new with revert option (users can switch to the old), and keep just the new.
Backwards compatibility. We should strive to maintain backwards compatibility when changing existing Translate interfaces or APIs. This requirement can be relaxed on a case-by-case basis when the interface is not used a lot.
MediaWiki. We may hit limitations of the MediaWiki platform. For example customizing the signup process is extremely inconvenient and may necessitate complicated core changes and also complicate compatibility requirements with stable MediaWiki releases. Work being done by the E3 team on the login page may ease the burden slightly.
Do not melt the schedule. The three tracks together make this project larger than the ULS, which has taken months to develop. It is likely that we need to prioritize heavily and make some hard choices on what features to drop. Further user testing and priorisation by Product Management will help to identify highest priorities.
Complexity. The workflows in Translate are much more varied and numerous than in ULS, which can lead to very high implementation complexity, which in turn is major source of bugs. We should aim to keep it simple and design for the most common use cases while not making rarer use cases impossible.
Sprinting up Niagara Falls. Use the just enough agile directive, release early, release often and incremental improvements. We must seamlessly work together and anticipate potential bottlenecks inside and outside our team. We must deliver something every two weeks and release something not complete, but usable, many times during the project.
Tracking. We want to measure how well the new designs compare against the existing ones. The current availability of tracking infrastructure needs to be researched and evaluated for our purposes. Implementation of tracking and measuring the results are not part of the milestones, only the requirements to what to measure.

Sign-up track (in progress)

The new design has multiple goals:

Start early. Translate as soon as possible.
Start easy. Guide and simplify first translations.

There are some technical risks in the implementation and limitations of reusability:

Design (especially of the frontpage) is not easily adaptable to other sites. This can partially be mitigated by documenting design decisions, so that the steps to come to a design can be repeated.
Sign-up process in MediaWiki is not easily customizable.
Sign-up process currently has a lot of spam protection and other security measures.
Need to implement sandboxing for unconfirmed translators.

Interest in reworking the backend and frontend signup process keeps popping up every once in a while. However, the current state is that there is not even a WebAPI for it. We need security review on our code. By dropping support for external authentication methods (LDAP, OpenID, Persona, Facebook, etc.) and thanks to the way the sandbox is proposed to be implemented, we can drop a lot of the complexity of the normal signup page. The only downside of the sandboxing is that it needs to restrict absolutely all write actions to the wiki before the users are confirmed. We also need security review for the sandbox code, although less urgent.

The original design did not address the administrator side of translator confirmation, but milestones have been added for that.

This track consists mostly of JavaScript and CSS work with some low level MediaWiki tinkering. Also some manual work is needed for creation of a list of messages presented to new translators.

Milestone SG1: poking with a stick (completed)

The first step is to figure out what is it exactly that we are going to do. What data and features do we need? What issues there are that might prevent implementing the original design? What help do we need outside of our team? How do we split the work into manageable pieces? This document is all about that, so continue reading.

User stories:

As a product manager I have a technical plan, so that I can start including stories in our sprints.
As a product manager I am aware of potential issues, so that I can make preemptive actions on them.

Deliverable:

This document

Milestone SG2: new frontpage (near completion)

Currently it is unclear whether this milestone will be included. For it to be included, more design work needs to be done to come up with good layout and information structure.

User stories:

As a translator I immediately get the idea what I can do on this website.
As a user, I can find information whether I can use translatewiki.net for my project.

Prerequisite:

Design complete.

Acceptance criteria:

The design will be kept in GitHub: https://github.com/planemad/Translate-UX/tree/gh-pages

Milestone SG3: sandbox (completed)

"Sandbox" in this context is not a playground area in the wiki, but a technical term describing an environment inside MediaWiki where the actions of new, unconfirmed translators will not affect the actual system. Aspiring translators will only be able to perform safe, whitelisted actions.

tule tule hyvä kakku, älä tule paha kakku
— Finnish children chant while making sand cakes

We can drop many spam protection measures from the user signup process by implementing a sandbox. While in a sandbox, users don't have write access to anything: let's call this a closed sandbox. All the sandbox user's actions are recorded in a dedicated place, so that we can easily get rid of spam accounts.

The plan is to register users in the users table and add them to a special user group sandboxed. Hooks will prevent any write actions by these users except those allowed. User acceptance is just a removal of the user from the special user group and an addition to the translator group.

This user group can be used to send reminder emails to translators and to purge unaccepted accounts automatically, depending on their age.

User stories (5 points):

Prerequisite for later milestones.
As a wiki administrator, my wiki is not spammed or vandalized by translators I have not confirmed, so that time is not consumed cleaning up the wiki.

Acceptance criteria:

Users in the special group cannot do any write actions
Documented, tested and reviewed utility class and WebAPI module:
1. Creating sandboxed users
2. Removing sandboxed users
Maintenance script (or special page) to purge by age
Maintenance script (or special page) to send reminders
Reminder emails that can be translated

Milestone SG4: signup dialog (completed)

A widget is needed to get the signup process started. I believe we can do this purely in JavaScript to simplify the implementation. The interaction with the backend happens through the API module written in the previous milestone. (I'm not saying that it is necessarily easier to do it via JavaScript, but this should avoid problems keeping backend HMTL generation and frontend JavaScript in sync and nice-and-tidy. It may introduce problems with placing the stuff in the page, but should allow smoother continuation to the next dialog.)

This widget will be alternative to the standard signup form, not a replacement.

User stories (5 points):

As a to-be translator I only need to fill as little information as possible to get into business.

Acceptance criteria:

Reviewed, tested and documented code
UI should have:
1. Language skill selection with suggestions
2. Mandatory info fields: username (with availability checking), email and password
3. See Translation_UX/Specification#Sign_up_dialog for further design specifications
Interacts with the backend to create sandboxed users

Milestone SG5: collection of simple messages for signup

We need a list of known simple or simplified messages, which have good documentation and suggestions. If time permits, we should also a collect list of suggestions as well, so that we can check how well they use good and known bad suggestions.

User stories (2 points):

Requirement for the simplified translation dialog

Acceptance criteria:

List of about 100 messages (should be based on messages available in translatewiki.net)
Different kind of messages (short, a bit longer)
Messages should have message documentation (can be fetched from live data)
With good and bad translation suggestions (existing translations are assumed to be good, but we don't want to always present them, nor are they always available)
Copes with missing data (translator for a previously unsupported language signs up) (explained somewhere)

Milestone SG6: simplified translation dialog

Gentle introduction to translators. Contains explanatory texts and some elements are not included. Depends on having the full editor being developed first.

User stories (3 points):

As a new translator, I can learn the translation editor (and process) step by step so that I don’t get overwhelmed and run away.

Acceptance criteria:

Layout as proposed (see attached image for details)
Supports only translation of the messages collected in the previous milestone
Translations do not affect live site
Limit reached notification if all messages are translated
Translatable email notifications of approval

Milestone SG7: translator approval page (completed)

The admins need an easy way to approve new translators. A dedicated page seems to be the path of least resistance.

Open issues:

A simple design should be proposed for this page, but faster implementation is preferred over better usability for now.
It should be decided what exactly to do with translators that are not spam, but who we do not want to confirm as-is.

User stories (3 points):

As a translation administrator I can easily accept new translators, because I can inspect their contributions.

Acceptance criteria:

Reviewed, documented and tested code
List users ordered by importance (for example number of translations made or signup date)
Allow to compare their translations to existing translations when available, otherwise just show them
Flag if bad suggestions were used
Show whether email address is confirmed
Few click promotion or decline
Nice to have: Way to send reminder for users who only made one or two translations which is not enough to confirm them

Milestone SG8: tracking

We want to know how well the new signup process works. Specific metrics to be defined. We need to observe:

Information about how the process is followed (e.g., number of new users failing the example translation, reaching the limit, discarded as spammers).
Overall impact (number of new users, users that remain active translators after a time).
Metrics of group 2 should be observed before and after deploying, to compare current and new system effects.

User stories (1 point):

As a designer I can see how well the implementation works so that I can point out issues and suggest enhancements.
As a developer I know what track points I need to implement.

Acceptance criteria:

List of things we want to measure
Document describing how implement necessary stuff to measure them (urls, clicks, whatever)

Translation track (completed)

The new design has multiple goals:

Fluency of translation. Support the common flow.
Clarity of information. Show the needed information.
Gradual conversion. Support evolution in expertise level.

This track has lots of components which can be tested simultaneously with the old implementation: message list, message editor, proofreading view.

Risks, limitations and special notes:

Large existing code base and feature set. Users are passionate of every single intended and unintended feature.
Need to collect more information to implement some features.
Tailored user experience based on collected information.
Need to care about performance in backend. We need to ensure our data collection and retrieval is fast enough.
Need to care about performance in the frontend.
Some possible high-hanging fruits that are likely to get dropped: threaded discussion and vertical diffs.

This track contains mostly JavaScript and CSS work alongside PHP work on the backend to provide relevant APIs.

Milestone TR1: poking with a stick (completed)

The first step is to figure out what is it exactly that we are going to do. What data and features we need? What issues there are that might prevent implementing the original design? What help do we need outside of our team? How do we split the work into manageable pieces?

User stories:

As a product manager I have a technical plan, so that I can start including stories in our sprints.
As a product manager I am aware of potential issues, so that I can make preemptive actions on them.

Acceptance criteria:

This document

Milestone TR2: new face for message list (completed)

ei pidä mennä merta edemmäs kalaan
— Finnish proverb

Let's begin by changing the current message list display. We will update the color scheme, remove message key names, add an edit link and add a label for fuzzy messages. This should work alongside with the old table and be accessible by user preference or request parameter.

User stories (2 points):

As an user I can experiment and try a feature rich and attractive translation interface, where the translation strings are listed for easy translation and navigation.

Acceptance criteria:

One message per line with truncation
Fuzzy indicator
Edit button
Color scheme as proposed
No message key displayed in the list
Way to beta-test this new layout for users
I18ned, works in RTL
Code is reviewed, tested and documented

Milestone TR3: progress bar (completed)

This little bar we will be needed in multiple places: group lists, message lists, different full-page views etc. It makes sense to make it as a reusable component.

User stories (1 point):

As a developer I can easily add a progress bar to my layout, so that I can do more other stuff.

Acceptance criteria:

Code is reviewed, tested and documented
Style configurable via CSS or equivalent
Possibility to choose which states to include (untranslated, outdated, translated, proofread)
Use it in the message list implemented in the previous milestone

Milestone TR4: project selector (completed)

The message list needs a new project selector. In this milestone we implement a widget to select project(s). It does not include integration to the message list. The project list works similar to language selector: keep track of recent or favorite projects, have search and have full list of projects. We need to start storing project icons in a machine accessible way.

User stories (3 points):

As a user I can quickly switch between projects.
As a developer I can integrate this widget into my layouts.

Acceptance criteria:

Code is reviewed, tested and documented
Flexible widget with PHP/JS APIs
I18ned, works in RTL
Has search
Can show favorite or recent projects
Data collection of last used projects
Use project icons when available
A way to specify the icons for each message group

Milestone TR5: selector integration (completed)

ei kahta ilman kolmatta
— Finnish proverb

Now that we have group selector, we can integrate it with the message list. Let's also add a language selector, which is expected to come from the Universal Language Selector. Also any work needed to make the task related selectors as per design.

User stories (2 points):

As a user I can finally change easily between projects, languages and tasks.

Acceptance criteria:

Multi-level project selector
Language selector
Restyle of other selectors

Milestone TR6: tracking (postponed)

Now that we have mostly completed the new message list implementation, it's about time we integrate user tracking for it. It's still open how we will implement the tracking, but we should come up with metrics we want to track and plan how to track them.

Specific metrics to be defined. We need to observe:

???
Usage of certain actions (like asking help) and whether they are completed.
Overall impact (number of translation/edits/proofread actions made).
Metrics of group 2 should be observed against the existing system.

User stories (2 points):

As a designer I can see how well the implementation works so that I can point out issues and suggest enhancements.
As a developer I know what track points I need to implement.

Acceptance criteria:

List of things we want to measure (everything related to translation track)
Document describing how implement necessary stuff to measure them (urls, clicks, whatever)

Milestone TR7: the new message editor is born (completed)

After the message list, the message editor is the next big task. It doesn't contain major changes in the contents, mostly just a restyling and rearranging of the information. The new editor version should be testable alongside with the old editor. There should be a user preference or another way to activate it.

The restyling of the translation helper boxes is not part of this story, though the relevant CSS styles and HTML layouts should be available.

User stories (3 points):

As a user I can start testing the new message editor.

Acceptance criteria:

Collapsible split view
Most message helpers move to right hand side
Use the proposed layout and colors where possible
1. Message key
2. Button styling
3. Ask question
Way to access the new editor

Milestone TR8: message editor in puberty (completed)

Now that we have the basic layout with stuff approximately in the correct place, we can start implementing the more detailed changes.

User stories (5 points):

As a user I can start using the new message editor to report bugs.

Acceptance criteria:

New shortcuts for actions (ctrl+s, ctrl+d etc...)
“This message needs translation in language X” in the message list
Display of message definition
Display of warnings (outdated, checkers)
Display of diffs (if not too time consuming)
Unsaved translation shown in message table
Collection of skipped translations

Milestone TR9: message editor with reformatted suggestions (completed)

The remaining task is to change the display of various suggestions. This also needs some code changes. To make shortcuts discoverable it was suggested to highlight the numbers (or letters) when Ctrl key is pressed.

Open issues:

Mock-up of uncollapsed tm/mt suggestions

User stories (3 points):

As a user I can use the clearer message editor to work faster.

Acceptance criteria:

Display of message documentation with edit link as per design
Translation memory and machine translation suggestions (collapsing the dots and with link to show them)
Display of In other languages
Shortcuts and links for suggestions to use them in translation
Any other tweaks needed to make the layout work for varying amount of content

Milestone TR10: full-page and proofreading view (completed)

This feature is mostly for page translation. Page translation generally has less info (suggestions, warnings, documentation) and longer texts. It needs less chrome and more space for seeing the context.

User stories (3 points):

As a translator I can see the approximate context of the messages so that I can use that information when translating.

Acceptance criteria:

Way to activate full-page view in message list
Original, compare and translated tabs
Progress bar
Navigation elements to jump to next message
Task and language selector
Compact indicators
Proofreading mode

Search track (completed)

The new design has multiple goals:

Avoid to think in advance. Avoid the need for users to understand the information structure to fix a typo.
Support quick connections. Quick access to actions and related information.

There are also some risks and limitations:

Limited by features in Solr
We might need to replace our existing Solr schema
Should solicit feedback from search experts like Oren
Small chance of issues with Solr on WMF cluster, old version, issues in Solarium, switch to ElasticSearch etc.

Milestone SR1: poking with a stick (completed)

The first step is to figure out what is it exactly that we are going to do. What data and features do we need? What issues are there that might prevent implementing the original design? What help do we need outside of our team? How do we split the work into manageable pieces?

User stories:

As a product manager I have a technical plan, so that I can start including stories in our sprints.

As a product manager I am aware of potential issues, so that I can make preemptive actions on them.

Acceptance criteria:

This document

Milestone SR2: mapping out (completed)

We need to figure out the schema for Solr. Can we extend the existing schema or do we need to develop a new one? What are the limitations and how to work around them?

User stories (2 point):

As a developer I know what information I need to fetch and submit to the search system, so that I can write code for that.

Acceptance criteria:

Draft schema which describes what data is needed and in which format
Short description how to handle updates and deletions of data

Milestone SR3: feed it (completed)

This task entails writing the code that updates the search system with translation data. This includes initial bootstrapping and updating, and deleting based on changes in the wiki. This milestone is deployable, which enables silent data gathering and creation of real search databases for measurements.

User stories:

Requirement for later milestones.

Acceptance criteria (3 points):

Code is unit tested and reviewed
Script that imports current wiki content to search system
Relevant changes in the wiki are reflected in the search system
Documented interfaces (PHP code) how to search and update the search system

Milestone SR4: use it (completed)

ei kysyvä tieltä eksy
— Finnish proverb

Implement a simple search interface using a new special page. It allows full-text searching providing a simple listing of search results with paging. This can be done with mock or real data; in latter case it depends on the previous milestone.

User stories (3 points):

As a user I can use a simple search interface so that I can find problematic translations and update them easily and provide feedback for further development.

Acceptance criteria:

Code is unit-tested and peer-reviewed to a reasonable extent
New special page: Special:TranslationSearch
The special page has input for fulltext search
Search will display matching results (the page content)
Search results are limited to one page
Translation editor is reachable by one click

Milestone SR5: facets (completed)

Time to implement some filtering. Let's start with language and project facets, continued by translation state (pending research).

Facet in short: facet is a classification that applies to search results. For example if you search for cars, you can use facets like color, number of doors or manufacturer to narrow your search. Each facet has a limited number of alternatives presented to the user. The number of results in each possible option is displayed before the user makes a choice.

User stories (3 points):

As a user I can narrow my search results so that I can more easily find the messages I am looking for.

Acceptance criteria:

Language facet in the left-hand side per design mockup
Project facet in the left-hand side per design mockup

Milestone SR6: tracking (postponed)

To get more information for open questions and further tweaks, we should be able to track search queries and the search results. For example whether we need to implement faceting of the message state (untranslated, outdated, translated). For this we can add server side logging of search queries and number of search results.

User stories (1 point):

As a designer I can see how well the implementation works so that I can point out issues and suggest enhancements.
As a developer I know what track points I need to implement.

Acceptance criteria:

List of things we want to measure
Document describing how implement necessary stuff to measure them (URLs, clicks, whatever)

Milestone SR7: polish (completed)

Make the search listing nicer as in the design. Collapse repeating messages and provide links to show them.

User stories (3 points):

As a user I don’t need to scan long lists of repeating messages to find what I’m looking for.

Acceptance criteria:

Collapsing of similar/identical search results
Links to expand the hidden results

Milestone SR8: cross integration (completed)

Integrate the editor developed in the translation milestone with the search results. Integrate the search page to message listing page.

User stories (2 points):

As a user willing to change several similar translations/strings, I don't need to leave the search results to update translations, so that I don't lose track of what I've already handled.

Acceptance criteria:

Editor to search page
1. Way to activate editor
2. Editor has all the usual features
3. Translation can be updated or added
Search to message list
1. Search goes to translation search page instead of normal search
2. Only one search box available (not both our own and the standard search)