Wikimedia Engineering/Report/2014/August/summary
 Summary for July 2014 |
Summary of the Wikimedia engineering report, August 2014 | Summary for September 2014 |
- This content is prepared for inclusion in the August 2014 Wikimedia Foundation report. It is a shorter version of the full Wikimedia engineering report for August 2014.
We migrated test.wikipedia.org to HHVM in early August and saw very few issues. Giuseppe shared some promising benchmarks. Re-imaging an app server was surprisingly painful, in that Giuseppe and Ori had to perform a number of manual actions to get the server up-and-running, and this sequence of steps was poorly automated. Doing this much manual work per app server isn't viable.
Mark submitted a series of patches to create a service IP and Varnish back-end for an HHVM app server pool, with Giuseppe and Brandon providing feedback and support. The patch routes requests tagged with a specific cookie to the HHVM back-ends. Tech-savvy editors were invited to opt-in to help with testing by setting the cookie explicitly. The next step after that will be to divert a fraction of general site traffic to those back-ends. The exact date will depend on how many bugs the next round of testing uncovers.
Tim is looking at modifying the profiling feature of LuaSandbox to work with HHVM; it is currently disabled.
In August, the Mobile Apps Team focussed on bug fixes for the recently released iOS app and for the Android app, as well as gathering user feedback from Wikimania.
The team also had unstructured time during Wikimania, in which the engineers are free to work on whatever they fancy. This resulted in numerous code quality improvements on both iOS and Android. On iOS, the unstructured time also spawned a preliminary version of the feature "Nearby", which lists articles about things that are near you, tells you how near they are to you, and points towards them. On Android, the unstructured time spawned a preliminary version of full text search, an improved searching experience which aims to present more relevant results.
This month the mobile web team, in partnership with the Editing team, launched a mobile-friendly opt-in VisualEditor for users of the mobile site on tablets. Tablet users can now choose to switch from the default editing experience (wikitext editor) to a lightweight version of VE featuring some common formatting tools (bold and italic text, the ability to add/edit links and references). We also began building a Wikidata contribution game in alpha that will allow users to add metadata to the Wikidata database (to start, occupations of people) directly from the Wikipedia article where the information is contained. We hope to graduate this feature to the beta site next month to get more quantitative feedback on its usage and the quality of contributions.
In August, the Flow team created a new read/unread state for Flow notifications, to help users keep track of the active discussion topics that they're subscribed to. There are now two tabs in the Echo notification dropdown, split between Messages (Flow notifications) and Alerts (all of the other Echo notifications). Flow notifications stay unread until the user clicks on the item and visits the topic page, or marks the item as read in the notifications panel. The dropdown is also scrollable now, and holds the 25 most recent notifications. Another new change -- subscribing to a Flow board gives the user a notification when a new topic is created on the board.
In August, the team working on VisualEditor presented at Wikimania 2014, worked with a number of volunteers at the hackathon, adjusted key workflows for template and citation editing, made major progress on Internet Explorer support, and fixed over 40 bugs and tickets.
Users of Internet Explorer 11, who we previously prevented from using VisualEditor due to some major bugs, will now be able to use VisualEditor. Support for earlier versions of Internet Explorer will be coming shortly. Similarly, tablet users browsing the site's mobile mode now have the option of using a mobile-specific form of VisualEditor. A greater range of VisualEditor editing tools on tablets, and availability of VisualEditor on phones as well as tablets, is planned for the future.
Improvements and updates were made to a number of interface messages as part of our work with translators to improve the software for all users, and VisualEditor and MediaWiki were improved to support highlighting links to disambiguation pages where a wiki or user wishes to do so. Several performance improvements were made, especially to the system around re-using references and reference lists. We tweaked the link editor's behaviour based on feedback from users and user testing. The deployed version of the code was updated three times in the regular release cycle (1.24-wmf17, 1.24-wmf18 and 1.24-wmf19).
The SUL finalisation team continues to work on building tools to support the finalisation. There are four ongoing streams of work, and the team is on track to have the majority of the work by the end of September.
The ability to globally rename users was deployed a while ago, and is currently working excellently!
The ability to log in with old, pre-finalisation credentials has been developed so that users are not inadvertently locked out of their accounts. From an engineering standpoint, this form is now fully working on our test environment. Right now the form uses placeholder text; that text needs to be 'prettified' so that the users who have been forcibly renamed get the appropriate information on how to proceed after their rename, and more rigorous testing should be done before deployment.
A form to globally merge users has been developed so that users can consolidate their accounts after the finalisation. From an engineering standpoint, this form is now fully working on our test environment. The form needs design improvements and further testing before it can be deployed.
A form to request a rename has been developed so that users who do not have global accounts can request a rename, and also so that the workload on the renamers is reduced. From an engineering standpoint, the form to request a rename has been implemented, and implementation has begun on the form that allows renames to rename users. Once the end-to-end experience has been fully implemented and tested, the form will be 'prettified'.
The project is getting close to Day 1 of a Wikimedia Phabricator production instance. For better overview and tracking, the Wikimedia Phabricator Day 1 project was split into three projects: Day 1 of a Phabricator Production instance in use, Bugzilla migration, and RT migration. Furthermore, the overall schedule was clarified. In the last month, Security/permission related requirements got implemented (granular file permissions and upload defaults, enforcing that policy, making file data inaccessible and not only undiscoverable). In upstream, Mukunda added API to create projects and Chase added support for mailing lists as watching users. Chase worked on and tested the security and data migration logic. Mukunda continued to work on getting the MediaWiki OAuth provider merged into upstream. Chase and Mukunda also worked on the Project Policy Enforcer action for Herald, providing a user-friendly dropdown menu to restrict ticket access when creating the ticket. A separate domain for user content was purchased. Chase also worked on the scripts to export and import data between the systems and support for external users in Phabricator and the related mail setup. Chase and Chad also took a look at setting up Elasticsearch for Phabricator.
In August, the work to improve MediaWiki's core front-end libraries continued on two fronts. The preparation for implementing of the request for comment on refactoring MediaWiki's skin system continued, with all skins moved out of MediaWiki and into their own repos, cleaning up the old shared skins infrastructure to a better location, and improvements to the ResourceLoader to support the improvements to the skins system. The second prong of work, to create a "MediaWiki" theme for OOjs UI, a toolkit used to compose complex widgets, progressed in collaboration with Design. This work has taken longer than anticipated due to delays in agreeing the complexities of user interactions but is on target to be completed soon, after which the toolkit will switch to this theme for all users. Additionally, work to share code between the OOjs and Mantle efforts continued, with the EventEmitter interface being ready to switch over to a single shared codebase.
We published a report on mobile trends expanding the data presented at the July 2014 Monthly Metrics meeting. We started work on referral parsing from request log data to study trends in referred traffic over time.
August was mostly a month of travel and vacation for the service team. We deployed a first prototype of the RESTBase storage and API service in Labs. We also presented on both Parsoid and RESTBase at Wikimania, which was well received.
Later in August, computer science student Hardik Juneja joined the team as a part-time contractor. Working from Mumbai, he dived straight into complex secondary index update algorithms in the Cassandra back-end. At the end of the month, design work resumed, with the goal of making RESTBase easier to extend with additional entry points and bucket types.