Wikimedia Technical Engagement/Goals

Technical Engagement's main focus in FY 2022/23 lies on making our existing services and programs stronger in order to increase sustainability of the services that we provide to the Wikimedia movement.

The majority of the work that Technical Engagement does is foundational work. This includes maintaining and evolving Wikimedia Cloud Services including support for Striker, Paws and Quarry (Cloud Services). It includes maintaining and evolving Toolhub and the Developer Portal, coordinating the Techblog, and managing service requests around the tech community metrics dashboards (Production or third party services). It encompasses evolving Wikimedia's technical documentation and providing support for users of Phabricator, Toolhub and Cloud Services (Technical Documentation and Support). Finally, it includes coordinating initiatives and programs like Small Wiki Toolkits, Hackathons, Coolest Tool Award, Outreach- and internship programs, contributions to other open source projects, or advocacy for best FOSS practices (Outreach, Community and Capacity Building).

Planning framework

• Strengthen foundational work to make our existing services and programs stronger (80%).
• Focus/Experiment: Space for experiments to explore ideas and cross-functional focus projects to move the needle in a specific field (max 20%).
• Increase synergy effects and reduce SPOFs through cross-training and shared responsibilities.
• Broad annual direction for the different services and initiatives. Multi-quarter timelines for multi-quarter initiatives. Detailed planning and goal setting in quarterly planning cycles.

Previous quarters

April-June 2023 (Q4) Cloud Services Wikimedia Cloud Services Release Toolforge Build Service - phab:T194332 (multi-quarter project) Release community beta phab:T267374  Done Multiple language stacks phab:T325799 Find a sustainable local storage solution for cloud-vps phab:T324729  Done Move off from icinga and introduce alertmanager phab:T328502 (multi-quarter project) Support 'unmanaged' projects in cloud-vps phab:T326818 Modernize openstack rbac phab:T330759  Done Move WMCS dashboards to grafana.wmcloud.org phab:T333568 Create CloudLB PoC in dev phab:T324992 (multi-quarter project) Upgrade Ceph Upgrade Ceph and Host OS phab:T306820 phab:T309789 Technical Support and Documentation Outreach, Community and Capacity Building Small wiki toolkits January-March 2023 (Q3) Cloud Services Implement Kubernetes as a Service (Magnum) - phab:T280792  Done Migrate PAWS to Magnum phab:T308873 Ensure alerting and on-call systems are sustainable and scalable - phab:T313444 (multi-quarter project) Enable alerts and pages from cloudvps/toolforge/paws/quarry - phab:T313031 Improve integration with WMCS of SRE cookbook automation - phab:T319401  Done Collaboration with Infrastructure Foundations team Finalize cloud-cumin setup and utilize for WMCS cookbook runs Release Toolforge Build Service - phab:T194332 (multi-quarter project) Release community beta phab:T267374 Upgrade ToolsDB - phab:T301949  Done Finish migration to new ceph backend Continue GridEngine deprecation (multi-quarter project) Deprecate GridEngine Docs phab:T326756  Done Planned Toolforge jobs improvements phab:T327254  Done Upgrade OpenStack  Done Upgrade cloud vps openstack to Openstack version Zed - phab:T323086 Upgrade Ceph Upgrade Ceph and Host OS phab:T306820 phab:T309789 Shut down final remaining bare-metal NFS servers phab:T301280  Done Update maintain-dbusers.py to operate in virtualized environment phab:T303663 Create CloudLB PoC in dev phab:T324992 (multi-quarter project) Collaboration with Infrastructure Foundations team Test implementing cloud load balancers in development  Done Clarify database as a service offerings  Done Update Documentation to clarify when to use Trove and when to use Toolsdb phab:T326754 Audit Toolsdb users for potential migrations to Trove phab:T291782 Research and decide on new Openstack deployment paradigm phab:T326758  Done In support of adding new features, and creating a sustainable, maintainable and repeatable cloud deployment Research supporting unmanaged Cloud-VPS projects phab:T326818  Done Identify first users, develop support for non-puppet managed VMs Analyze and share results from Cloud Services Annual Survey - phab:T322500  Done Transform and massage the data Analyze the survey responses Publish and present the findings Production or third party services Improve Toolhub record content coverage and consistency - Phab:T327558 Collaborate with the local community to improve documentation of tools which support the workflows of Wikimedia Commons - Phab:T327559 Implement a data feed for Toolhub based on descriptions of tools on commonswiki - Phab:T327560 Mentor two Outreachy interns in the project “create a web application for editing Toolhub records” (workboard) Increase the number of tools that adhere to our published best practices [Tool Sweep; review at least 30 tools each month]. - Phab:T327398 Technical Support and Documentation Create on-wiki content based on Small Wiki Toolkits workshops conducted in 2022 - Phab:T327282 Prepare and migrate content of the first workshop: How to host a bot on Toolforge - Phab:T327283 Prepare and migrate content of the second workshop: How to run a basic script via Pywikibot - Phab:T327285 (stretch goal) Improve Cloud Services documentation - Phab:T321582 Finish and publish Trove user guide - Phab:T326854 Design new IA for Cloud docs and get feedback - Phab:T327319 Improve Pywikibot documentation - Phab:T320625 Pywikibot Toolforge tutorial - Phab:T134495 Cleanup and improvements as per Phab:T308063 Add features to documentation (collection) metrics tool - Phab:T320626 Define requirements for this tool Provide bugwrangler support in Phabricator Outreach, Community and Capacity Building Organise and run Wikimedia Hackathon 2023 Manage logistics for the event Coordinate scholarship application review and selection Design program framework Plan and run GSOC and Outreachy (summer round) - Phab:T326991 Recruit projects and mentors for upcoming GSOC/Outreachy rounds Support 2023 Wikimania Hackathon program planning Research and plan for integrating the Tech Blog with Diff - Phab:T327323 Complete stakeholder conversations and create a needs assessment and recommendation for the team Create a project plan for integrating the blogs Launch tech swag program - Phab:T320768 Finalize the design Submit the purchase order Improve our understanding of the tool ecosystem and tool maintainers' needs - Phab:T323302 Complete 4 interviews with community members and stakeholders - Phab:T323317 Investigate how to aggregate information about tools that is captured by manual review during Tool Sweep - Phab:T327397 October-December 2022 (Q2) Cloud Services Upgrade OpenStack Upgrade cloud vps openstack to Openstack version Yoga - phab:T305828  Done Implement Kubernetes as a Service (Magnum) - phab:T280792 (multi-quarter project) Migrate PAWS/Quarry to Magnum phab:T308873, phab:T301469 (Done in dev) Ensure alerting and on-call systems are sustainable and scalable - phab:T313444 (multi-quarter project) Enable alerts and pages from cloudvps/toolforge/paws/quarry - phab:T313031 Move cloud hosts to single NIC implementation - phab:T319184 Improve integration with WMCS of SRE cookbook automation - phab:T319401  Done Collaboration with Infrastructure Foundations team Finalize and Publish cross-realm guidelines for case 4 - phab:T296411  Done Release Toolforge Build Service - phab:T194332 (multi-quarter project) Implement Harbor (status: implemented in toolsbeta) Plan Beta Features and Delivery Upgrade ToolsDB - phab:T301949 Allow self-service tool deletion - phab:T285403  Done Continue GridEngine deprecation (multi-quarter project) Provide support for Toolforge jobs bugs/feature requests  Done Coordinate Cloud Services User Survey Setup Cloud Services User Survey in Lime Survey  Done Run Cloud Services User Survey  Done Production or third party services Finalize taxonomy model for Toolhub - phab:T308030  Done Implement controlled vocabulary tags in Toolhub Technical Support and Documentation Provide continuous support in Phabricator See also: Bugwrangler Building community, social connection, and technical expertise around documentation (multi-quarter project) Improve documentation and update Phabricator tasks around documentation in two collections (Pywikibot - phab:T320625; Cloud Services - phab:T321582) Publish "Intro to the Wikimedia Technical ecosystem" draft as a wiki page and as a guided experience for newcomers.  Done Improve mw:Documentation landing page and content - phab:T277128  Done Outreach, Community and Capacity Building Coordinate the Coolest Tool Award - phab:T313443 (multi-quarter project) Conduct the 2022 edition of the Coolest Tool Award  Done Coordinate Wikimedia's participation in Outreachy Round 25 and provide mentorship in one project (multi-quarter project) Prepare and support mentors and projects for Outreachy Round 25 - phab: T313361 Support Outreachy applicants and mentor an intern in the project "Create a Web Application for Editing Toolhub Records" Conclude the 2022 Small Wiki Toolkits workshops Organize two small wiki toolkit workshops  Done Reflect and summarize the past season of Small Wiki Toolkits and propose a plan for next year  Done Reward volunteer and mentor contributions through the yearly swag program (multi-quarter project) Create project plan for swag program Clarify responsibilities around Gerrit privilege handout workflow Clarify who needs to be involved/consulted on making the decision - Phab:T273164 Update the Gerrit Privilege Policy pages - Phab:T237618 Focus project Follow up of the experiment week projects in Q1 (planned start date: November, more info tba). July-September 2022 (Q1) Cloud Services Deprecate Gridengine (continues, multi-quarter project) Migrate tools off GridEngine - phab:T313405 (continues, multi-quarter project) Migrate at least one tool to k8s  Done Figure out if migrations of “like” tools can be templated and migrated more easily  Done Ensure alerting and on-call systems are sustainable and scalable - phab:T313444 (continues, multi-quarter project) Consolidate Alerting to a single platform (alertmanager)  Done Audit alerts to ensure any pages are required, as opposed to automatically filing a ticket, or simply removing  Done Release Toolforge Build Service - phab:T194332 (continues, multi-quarter project) Implement Harbor (status: implemented in toolsbeta) Plan Beta Features and Delivery Update Striker to work with GitLab for Toolforge tool git hosting automation Replace the existing Diffusion (Phabricator) git integration - phab:T296893  Done Migrate ~450 existing tool owned git repositories to GitLab - phab:T315706  Done Implement Kubernetes as a Service (Magnum) - phab:T280792 (continues, multi-quarter project) Finish bringing service online Evaluate usage for Quarry/PAWS - phab:T308873, phab:T301469  Done Evaluate usage for community use  Done Production or third party services Finalize taxonomy model for Toolhub - phab:T308030 Conduct community feedback round on taxonomy model  Done Implement controlled vocabulary tags in Toolhub (postponed to Q2) Get more people to understand tech community metrics on wikimedia.biterg.io and increase redundancy across the team Create onboarding plan for cross-training a second person - Phab:T294870  Done Organise a consultancy hour with Bitergia - Phab:T306787  Done Increase the number of blog posts on the Tech Blog - Phab:T313072 Draft & publish 4 blog posts from the Technical Engagement team Help 4 WMF staff members to draft and publish blog posts Help 4 technical contributors to draft and publish blog posts Technical Support and Documentation Make it easier to assess the state of documentation and contribute meaningful improvements (project board) Identify documentation collections and how to assess the state of the content they contain.  Done Make the results of this assessment publicly available and easy to navigate. (tracking: Phab:T313041) Complete basic content review and improvements for all on-wiki collection landing pages that are linked from the Developer Portal (Tracking: Phab:T312997) Review landing pages for m:Data dumps and Research:Data.  Done Review Quarry landing page.  Done Review Pywikibot landing page.  Done Provide continuous support in Phabricator See also: Bugwrangler Outreach, Community and Capacity Building Coordinate Wikimedia's participation in Outreachy Round 25 (continues, multi-quarter project) Recruit mentors and projects for Outreachy Round 25 - Phab:T313361  Done Continue with 2022 Small Wiki Toolkits workshops Organize three small wiki toolkit workshops on the topic of bots and scripts - Phab:T312612  Done Clarify Technical Engagement support on Rapid Grants applications - Phab:T304380 (postponed) Setup criteria for what makes a successful grant/project Decide on team capacity and workflow process to assess and review grant applications Clarify responsibilities around Gerrit privilege handout workflow (continues) Clarify who needs to be involved/consulted on making the decision - Phab:T273164 Update the Gerrit Privilege policy pages - Phab:T237618 Coordinate the Coolest Tool Award- phab:T313443 (continues, multi-quarter project) Plan the 2022 edition of the Coolest Tool Award Coordinate the 2022 Wikimania Hackathon - Phab:T313554  Done Define timeline and project plan Clarify timeline and integration in Wikimania program and platform Define project plan including main deliverables, support needs, and weekly tasks Implement the event Technical contributors have opportunities to share knowledge and connect socially Present clear resources for individual newcomers (and newish-comers) so they can find the information they need to build what they want. - Phab:T313629 Coordinate community-run sessions and spaces for people to share technical knowledge. - Phab:T313627 Coordinate community-run social sessions and spaces with fun activities. - Phab:T313627 Growing communities of technical contributors can find support and feel welcome at the Wikimania Hackathon Engage with local meetup and affinity meetup organizers from the Wikimedia Hackathon. - Phab:T313625 Ensure that the accessible platform and accommodations are advertised widely. - Phab:T313627 Technical contributors have opportunities to work together and to share outcome with Wikimania attendees Organize the pre-event showcase to help people connect with projects and people. This will include explicit instructions to use Phabricator and come back for the final showcase. - Phab:T313631 Design an accessible and useful platform space for people to collaborate in. - Phab:T313630 Provide Wikimania attendees an opportunity to learn about the hackathon projects at the final showcase. - Phab:T313631 Experiment Week We've planned in a few days this quarter to explore two topics:  Done Research how other open source communities support newcomers - phab:T313543 Create comparison matrix and summarize key findings Provide recommendation on what Wikimedia could try out or explore further  Done Identify metrics to measure the impact of a tool - phab:T313546 Research and summarize available data sources and gaps to define tool impact metrics Create visualization to display raw data