Wikimedia Engineering/Report/2014/June
Note: We're also providing a shorter, simpler and translatable version of this report that does not assume specialized technical knowledge.
Engineering metrics in June:
- 151 unique committers contributed patchsets of code to MediaWiki.
- The total number of unresolved commits went from around 1440 to about 1575.
- About 14 shell requests were processed.
Upcoming events
[edit]There are many opportunities for you to get involved and contribute to MediaWiki and technical activities to improve Wikimedia sites, both for coders and contributors with other talents.
For a more complete and up-to-date list, check out the Project:Calendar.
Date | Type | Event | Contact |
---|---|---|---|
8 July 2014 | ECT: Office Hour, 1600 UTC, #wikimedia-office | qgil | |
10 July 2014 | IRC discussion of frontend and UX standardization , 22:30-23:30 UTC | Trevor Parscal | |
15 July 2014 | Tech Talk: Hadoop and Beyond. An overview of Analytics infrastructure at 17:00 UTC, IRC: #wikimedia-dev | NRuiz (WMF) (talk) | |
16 July 2014 | IRC discussion of the Vertical writing support proposal , 1800-1900 UTC | Yair rand | |
23 July 2014 | IRC discussion of the proposal that we support Composer-managed libraries on the WMF cluster , 2100-2200 UTC | Bryan Davis | |
29 July 2014 | Tech Talk: HHVM in production: what that means for Wikimedia developers at 19:00 UTC, #wikimedia-dev connect | Ori.livneh (talk) | |
30 July 2014 | IRC discussion of Requests for comment/CentralNotice Caching Overhaul - Frontend Proxy , 22:00-23:00 UTC in #wikimedia-office connect. | Matt Walker |
Personnel
[edit]Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.
- VP of Engineering
- Software Engineer - Front-end (VisualEditor)
- Software Engineer - Services
- Software Engineer - Internationalization
- Software Engineer - Front-end
- Software Engineer - Mobile - iOS
- QA Tester
- Software Engineer - Full Stack
- Product Manager
- Product Manager - Language Engineering
- Operations Security Engineer
- Project Coordinator - Engineering
- Partner Manager - Wikipedia Zero (Latin America)
Announcements
[edit]- Elliot Eggleston joined the Wikimedia Foundation as a Features Engineer in the Fundraising-Tech team (announcement).
Technical Operations
[edit]New Dallas data center
- On-site work has started in our new Dallas (Carrollton) data-center (
codfw
). Racks have been installed, the equipment we moved from Tampa has been racked and cabling work has been mostly completed over the course of the month. We are now awaiting the installation of connectivity to the rest of our network as well as the arrival of the first newly-ordered server equipment, so server & network configuration can commence.
Puppet 3 migration
- In July we migrated from Puppet 2 to Puppet 3 on all production servers. Thanks to the hard work of both volunteers and Operations staff on our Puppet repository in the months leading up to this, this migration went very smoothly.
Labs metrics in June:
- Number of projects: 173
- Number of instances: 424
- Amount of RAM in use (in MBs): 1,741,312
- Amount of allocated storage (in GBs): 19,045
- Number of virtual CPUs in use: 855
- Number of users: 3,356
Wikimedia Labs
- Last month we switched the Labs puppetmaster to Puppet 3; this month all instances switched over as well. Some cleanup work was needed in our puppet manifests to handle Trusty and Puppet 3 properly; everything is fairly stable now but a bit of mopping up remains.
Editor retention: Editing tools
[edit]We added CSS styling to the HTML to ensure that Parsoid HTML renders like PHP parser output. We continued to tweak the CSS based on rendering differences we found. We also started work on computing visual diffs based on taking screenshots of rendered output of Parsoid and PHP HTML. This initial proof-of-concept will serve as the basis of more larger scale automated testing and identification of rendering diffs.
The GSoC 2014 LintTrap project saw good progress and a demo LintBridge application was made available on wmflabs with the wikitext issues detected by LintTrap.
We also had our quarterly review this month and contributed to the annual engineering planning process.Core Features
[edit]In June, the Flow team finished an architectural re-write for the front-end, so Flow will be easier to keep updating in the future. This will be released to MediaWiki.org the first week of July, and Wikipedia the following week.
The new feature in this release is the ability to sort topics on a Flow board. There are now two options for the order that topics appear on the board: you can see the most recently created threads at the top (the default), or the most recently updated threads. This new sorting option makes it easier to find the active conversations on the board.
We've also made a few changes to make Flow discussions easier to read, including: a font size now consistent with other pages; dropdown menus now easier to read; the use of the new button style, and the WikiGlyphs webfont.Growth
[edit]Additionally, the team released a major refactor of the GuidedTour extension's API, as well as design enhancements like animations, a new CSS-based way of drawing guider elements, updated button styles, and more. The team also launched GuidedTours on three new Wikipedias: Arabic, Norwegian, and Bengali.
Support
[edit]Core features of the app include the ability to save pages for offline reading, a record of your browsing history, and the ability to edit either as a logged in user or anonymously. Therefore the app is the first mobile platform that allows anonymous editing! The app also supports Wikipedia Zero for participating mobile carriers.
Additional work done this month includes the start of implementing night mode for the Android app (by popular demand), creating an onboarding experience which is to be refined and deployed in July, and numerous improvements to the edit workflow.Additionally, the team enabled downsampled thumbnails for a live in-house Wikipedia Zero operator configuration, and finished Wikipedia Zero minimum viable product design and logging polish for the Android and iOS Wikipedia apps. The team also supported the Wikipedia apps development with network connection management enhancements in Android and iOS, with Find in page functionality for Android, and response to Wikipedia for Android Google Play reviews.
The team facilitated discussions on proxy and small screen device optimization, and examined the HTML5 app landscape for the upcoming fiscal year's development roadmap. The team also created documentation for operators for enabling zero-rating with different connection scenarios. Bugfixes were issued for the mobile web Wikipedia Zero and the Wikipedia for Firefox OS app user experience.
Routine pre- and post-launch configuration changes were made to support operator zero-rating, with routine technical assistance provided to operators and the partner management team to help add zero-rating and address anomalies. Finally, the team participated in recruitment for a third Partners engineering teammate.Wikipedia Zero (partnerships)
- We launched Wikipedia Zero with Airtel in Bangladesh, our third partner in Bangladesh, and our 34th launched partner overall. We participated in the Wiki Indaba conference, the first event of its kind to be held in Africa. The event, organized by Wikimedia South Africa, brought together community members from Tunisia, Egypt, Ghana, Kenya, Namibia, Nigeria, Ethiopia, Malawi and South Africa. The attendees shared experiences and challenges to work in the region and formulated strategies to support and strengthen the movement's efforts across the continent. While in South Africa, Adele Vrana also met with local operators. Meanwhile, Carolynne Schloeder met with numerous operators and handset manufacturers in India. Carolynne joined Wikimedian RadhaKrishna Arvapally for a presentation at C-DOT, and both participated a blogger event hosted by our partner Aircel, along with other members of Wikimedia India in Bangalore. Smriti Gupta joined the group as Mobile Partnerships Manager, Asia.
- The Translate extension received numerous bug fixes, including fixing workflow states transitions for fundraising banners.
- Changes in input methods: Ludic (lud) transliteration and Tibetian (bo) EWTS layouts added.
MediaWiki Core
[edit]- Completing the necessary engineering work to carry out the finalisation.
- Setting a date on which the finalisation will occur (Note: this date may be after September).
- Have a communications strategy in place, and community liaisons to carry that out, for the time period between the announcement of the date of the finalisation and the finalisation proper.
Security auditing and response
Quality assurance
[edit]Quality Assurance/Browser testing
/qa/browsertest
repository either to /mediawiki/core
or to their relevant extension. This gives us the ability to package browser-based acceptance tests with the release of MediaWiki itself. After more than two years evolving the browser testing framework across WMF, the /qa/browsertests
repository is retired, and all if its functions now reside in the repositories of the features being tested.Multimedia
[edit]This month, we started working on the Structured Data project with the Wikidata team, to implement machine-readable data on Wikimedia Commons. We are now in a planning phase and aim to start development in Fall. We ramped up our work on UploadWizard, reviewed user feedback, collected metrics, fixed bugs and started code refactoring, with the help of contract engineer Neil Kandalgaonkar. We also kept working on technical debt and bug fixes for other multimedia tools, such as image scalers, GWToolset and TimedMediaHandler, with the help of Summer contractor Brian Wolff.
As product manager, Fabrice Florin helped plan our next steps, hosting a planning meeting and other discussions of our development goals, and led an extensive review of user feedback for Media Viewer and UploadWizard with new researcher Abbey Ripstra. Community liaison Keegan Peterzell introduced Media Viewer and responded to user comments throughout the product's worldwide release. To learn more about our work, we invite you to join our discussions on the multimedia mailing list.- Tools for mass migration of legacy translated wiki content
- Wikidata annotation tool
- Email bounce handling to MediaWiki with VERP
- Google Books, Internet Archive, Commons upload cycle
- UniversalLanguageSelector fonts for Chinese wikis
- MassMessage page input list improvements
- Book management in Wikibooks/Wikisource
- Parsoid-based online-detection of broken wikitext
- Usability improvements for the Translate extension
- A modern, scalable and attractive skin for MediaWiki
- Automatic cross-language screenshots for user documentation
- Separating skins from core MediaWiki
- Chemical Markup support for Wikimedia Commons
- Improving URL citations on Wikimedia
- Historical OpenStreetMap
- Welcoming new contributors to Wikimedia Labs and Tool Labs
- Evaluating, documenting, and improving MediaWiki web API client libraries
- Feed the Gnomes - Wikidata Outreach
- Template Matching for RDFIO
- Switching Semantic Forms Autocompletion to Select2
- Catalogue for MediaWiki Extensions
- Generic, efficient localisation update service.
Volunteer coordination and outreach
- 2014-06-02 — Discussion of Requests for comment/Grid system;
- 2014-06-11 — Discussion of several RfCs (lightning round);
- 2014-06-13 — Discussion of security guidelines draft;
- 2014-06-20 — Discussion of revamping MediaWiki's skin systems: Trevor Parscal's "Redo skin framework" and Bartosz Dziewoński's "Separating skins from core MediaWiki" work;
- 2014-06-25 — Discussion of front-end and UX standardization.
Analytics/Editor Engagement Vital Signs
We analyzed the early impact of the tablet desktop-to-mobile switchover on traffic, edit volume, unique editors, and new editor activation.
We hosted the June 2014 edition of the research showcase with two presentations on the effect of early socialization strategies and on predictive modeling of editor retention.
We released wikiclass, a library for performing automated quality assessment of Wikipedia articles.
We released longitudinal data on the daily edit volume for all wikis with VisualEditor enabled, since the original rollout.
We continued work on an updated definition for PageViews.
Finally, we held our quarterly review (Q4-2014) and presented our goals for the next quarter (Q1-2015).The Wikidata project is funded and executed by Wikimedia Deutschland.
- The team worked on fixing bugs as well as a number of features. These include data access for Wikiquote, support for redirects, the monolingual text datatype as well as further work on queries. Interface messages where reworked to make them easier to understand. First mockups of the new interface design have been published for comments. The entity suggester a team of students worked on over the last months has been deployed. This makes it easier to add new statements by suggesting what kind of statements are missing on an item. Wikidata the Game has been extended by Magnus by 2 games to add date of birth and date of death to people as well as to add missing images.
Future
[edit]- The engineering management team continues to update the Deployments page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the annual goals, listing ongoing and future Wikimedia engineering efforts.