Extension:Translate/Mass migration tools/Project updates

Project title: Tools for mass migration of legacy translated wiki content

Pre-Community Bonding Period

Read the existing documentation related to Translate extension

Documentation for Extension:Translate

Translators (main help page )

Translation administrators

Sysadmins and developers

Translate this template

Prepared pages for translation and imported them manually
Went through the coding conventions for the language selected - JavaScript
Set up the coding environment - editor, gerrit ssh keys, etc. Ready to submit patches

Community Bonding Period Report

Goals for first half

As mentioned in the project schedule, the first half of the project aims to accomplish the following goals -

Coding of the confirmation and publishing screens related to Step 2 (importing translations), with basic initial import of old text is to be done by May 28
Once the initial import is done, enhancement bugs would be filed and fixed in the next few days until June 7
Desiging and coding of the algorithm for step 1 (preparing page for translation) would be done by June 23

Communication plan

IRC has been used so far for daily communication and mails when the mentors were not available
It has been decided to have daily meetings except on Sunday's at #mediawiki-i18n from 13.45 UTC to 14.00 UTC

Lessons learnt

Not to get stuck on a particular task and keep things simple for the first version
Strict use of Gerrit for code review
Second part of the project is more important and shall be done first

Work done

April 22 to May 5

Documented the requirements for the project
Designed an algorithm for aligning the translations
Wrote a script to fetch the source translations and align the section headers

May 6 to May 18

As per the meetings held, it has been decided that step 2 be done before step 1
The second step would be seen as a special page - Special:PageMigration. A skeleton of the same was set up on local wiki
Submitted patch sets for the first version of the special page. As of now, it -

Fetches and displays the translation units created by Translate for the given page
Imports the old translations for given language on paragraph level and displays them
Feature can be now conditionally enabled by setting $wgTranslatePageMigration to true

Screenshot showing the same

Weekly Reports

Week 1: May 19 to May 25

Added features to edit, delete and swap the imported translations - Patch
i18n of messages used so far - Patch

Week 2: May 26 to June 2

Added feature for manual splitting of units by giving an option to add an empty unit after the current one - Patch
Created an instance on wmflabs and the tool can now be tested here.
Restricted access to the Special page to translation admins only on Wikimedia wikis - Patch
Fixed the IDs and class names adapting the BEM approach and revamped the interface using the MediaWiki grid layout - Patch

Week 3: June 3 to June 9

Wrote QUnit tests for the various API calls involved - Patch
Handled possible error cases like non-existence of a page, absence of old translations, etc - Patch
Added hover text for image icons to convey their meaning - Patch
Fixed the overall code architecture - Patch
Wrote a module for simplistic alignment based on h2 headers - Patch

Week 4: June 10 to June 16

Handled whitespace in input at Special:PageMigration by trimming them - Patch
Started with Step 1 of the project - preparing the page for translation
Wrote a script under Translate itself which does the basic work of preparing the page for translation. It includes adding ‎<translate> tags at the start and end of the page, adding newlines before and after section headers, handling of files, internal links as per the markup examples and saving of the page with the modified text - Patch
Handled internal links to sections on same page by linking them to language-independent anchor - Patch

Week 5: June 17 to June 23

Handled templates by keeping them as a part of the page template - Patch
Continued to work on the internal links to sections on same page patch

Week 6: June 24 to June 30

New icon images were introduced at Special:PageMigration - Patch
A new special page Special:PagePreparation was introduced which serves as the confirmation screen for step 1 - Patch

Week 7: July 1 to July 7

No work done due to relocation to Hyderabad

Week 8: July 8 to July 14

Continued to work on Special:PagePreparation. Added feature to show diff and save the page on user confirmation

Week 9: July 15 to July 21

Addressed code review for the page preparation patch
Addressed code review by Nikerabbit and Siebrand for Special:PagePreparation patch
Addressed code review for the categories related patch

Week 10: July 22 to July 28

Got the page preparation patch merged by removing the file handling part and addressing other issues.
Continue addressing CR for categories patch
Addressed code review for internal links to sections on same page patch
Added 'translation' as magic word under the Translate extension - Patch
Files (Images) were handled as a part of page preparation. The regex was built by first fetching the aliases of 'File' namespace. - Patch
i18n, L10n issues for Special:PagePreparation were addressed.
Punctuation added for error messages used at Special:PageMigration - Patch

Week 11: July 29 to August 4

Addressed code review for internal links to sections on same page patch
Got Special:PagePreparation patch merged by addressing the remaining issues
Addressed code review for translation as magic word patch
Changed 'translation' as magic word to '#translation' as parser function
Addressed code review for the new files with namespace aliases patch

Week 12: August 5 to August 11

Addressed code review for '#translation' as parser function patch
Addded the mw-ui-input class for input field at Special:PagePreparation - Patch
New icon images for Special:PageMigration patch was merged
Changed api call changed from .get to .post for the getDiff() function at Special:PagePreparation - Patch
Skipped source units on pressing tab key at Special:PageMigration - Patch
Addressed remaining issues for files patch and got it merged
Splitting of units was done after the h2 alignment - Patch
Amended the #translation as parser function patch to handle the case of source page or invalid language code
Submitted a patch to show messages after 'Import' and 'Save' at Special:PageMigration - Patch
Made input fields at both the special pages bigger and usable - Patch
Normalized page title used at Special:PageMigration - Patch
Submitted a fix to unwanted pixel for switch images - Patch

Week 13: August 12 to August 18

Wrote parser tests for the #translation as parser function patch
Addressed code review for page title normalized patch and got it merged
Addressed code review for input fields patch
Discussions and testing of split headers after alignment patch
Rebased and submitted to categories patch which uses #translation as parser function and namespace aliases from the files patch
A cancel button was added to Special:PagePreparation to abort the process - Patch
Relative links were ignored while adding the 'Special:MyLanguage/' prefix - Patch
Edit summary was changed to appear in content language in case of both the special pages - Patch
Added new feature to split headers from other wikitext on unit level - Patch
Did some testing of both the special pages on mediawiki.org
Wrote help pages for Special:PagePreparation and Special:PageMigration