Jump to content

Content translation/Deployments/Deployment Plan/Archive4

From mediawiki.org

CX Deployment Plan for January 2015

[edit]
  • Project: Content Translation
  • Deployment date: Week of January 12, 2015
  • Initial Release: Released in Beta Labs environment (July 2014)
  • Past Releases: See: Archives
  • What is targeted for January 2015 release:
Content Translation 0.03 release: https://www.mediawiki.org/wiki/Content_translation/Roadmap#Ongoing_Release:_Content_Translation_0.03_release
  • Long-term project roadmap:
https://www.mediawiki.org/wiki/Content_translation/Roadmap
  • Language Pairs to be supported:
Target Wikipedia Sources (MT) Sources (without MT)
Catalan Spanish, Portuguese, English -
Spanish Catalan, Portuguese, English -
Portuguese Catalan, Spanish English
Danish - English, Swedish
Esperanto English -
Malay Indonesian English
Indonesian Malay English
Norwegian (Bokmål) Norwegian Nynorsk English
  • List of wikis to deploy Content Translation
    • cawiki
    • dawiki
    • eowiki
    • eswiki
    • idwiki
    • mswiki
    • nnwiki
    • nowiki
    • ptwiki
    • svwiki
  • Release as: Beta Feature

Overall Plan

[edit]

This release is targeted to Production with 0.03 feature set.

System Architecture

[edit]

See:

https://www.mediawiki.org/wiki/Content_translation/Technical_Architecture
https://www.mediawiki.org/wiki/Content_translation#Workflow_and_Technical_Architecture
https://www.mediawiki.org/wiki/Content_translation

Caching Architecture

[edit]

The following diagram includes the caching requirements for the CX framework:

https://www.mediawiki.org/wiki/Content_translation/Server_communications_workflow
https://commons.wikimedia.org/wiki/File:CX_ArchitectureV1.svg

Components to be provisioned for production

[edit]
cxserver
[edit]

Content Translation service is puppetize and available in operations/puppet repository as "cxserver" module.

Primary setup instructions are at: https://git.wikimedia.org/markdown/mediawiki%2Fservices%2Fcxserver.git/HEAD/README.md

For detailed information about component, installation and configuation and instructions: https://www.mediawiki.org/wiki/Content_translation/Setup

Apertium
[edit]

Apertium service is puppetize and available in operations/puppet repository as "apertium" module.

Apertium service installation and configuration: https://www.mediawiki.org/wiki/Content_translation/Apertium/Service

Node.js
[edit]
Dictd
[edit]

No additional configuration needed for the planned languages. In future, when we enable more languages, dictd based dictionaries or external dictionaries may get introduced- but not planned right now. See: https://www.mediawiki.org/wiki/Content_translation/Dictionaries for details.

Backend Services
[edit]
Extension dependencies
[edit]
  • BetaFeatures
  • CLDR
  • EventLogging
  • GuidedTour
External APIs called by CX
[edit]
  • Wikidata
  • Parsoid API
Configuration Scripts
[edit]

Provisioning Plan

[edit]

a. Storage and Hardware Requirements

Service cluster A.
  • cxserver service hardware is ready. For ContentTranslation hardware measurement, we monitored Beta cluster at,
https://tools.wmflabs.org/nagf/?project=deployment-prep#h_deployment-cxserver03
  • Apertium service hardware is ready. For Apertium hardware measurement, we monitored Beta cluster at,
https://tools.wmflabs.org/nagf/?project=deployment-prep#h_deployment-apertium01

b. Bandwidth Requirements

Service cluster A.
Tech Ops has indicated that cxserver/apertium instances will go into "Service cluster A" (ie http://ganglia.wikimedia.org/latest/?r=hour&cs=&ce=&c=Service+Cluster+A+eqiad&h=&tab=m&vn=&hide-hf=false&m=cpu_report&sh=1&z=small&hc=4&host_regex=&max_graphs=0&s=by+name )

d. Performance expectations

See, https://www.mediawiki.org/wiki/Content_translation/Performance

Monitoring and metrics

[edit]

LE Team responsibilities

[edit]
  • Kartik - Deployment, Engineer
  • Niklas - Engineer, Code Reviewer
  • Santhosh - Engineer, Code Reviewer
  • David - Engineer, Code Reviewer
  • Joel - Engineer, Code Reviewer
  • Runa - Team Scrum-Master / testing and communications
  • Pau - Feature UX reviewer, designer
  • Amir - Feature signoff

Timeline

[edit]
Date Milestone People Involved Status
December 12, 2014 Beta-Feature Sign Off Amir, Greg, JamesF Yes Done
December 12, 2014 Hardware Availability for CX and other services Tech Ops, Kartik (LE) Yes Done
Within 7 days of HW procurement Puppet config preparation for cxserver and apertium service Kartik (LE) Yes Done
Within 10 days of HW procurement Database deployment Kartik (LE), Tech Ops Yes Done
January 5-8, 2015 MediaWiki config and CX extension deployment Kartik (LE), Niklas, Ops, Greg, <any others> Yes Done
January 5-12, 2015 Beta Feature security review Chris S/Joel S Yes Done
Week of January 12, 2015 Deployment as beta-feature completed on 8 target Wikipedias Kartik (LE), <any others> Yes Done
January 13-16, 2015 Beta-feature testing period LE team, <any others> Yes Done

Current Status

[edit]

CX Deployment Plan for January 2015

Done

[edit]
  • Hardware Availability for CX and other services. Yes Done
    • Started: 2014-11-10
    • cxserver hardware.
      • Ready as per Ops/Alex. Planned to host on Service Cluster A.
    • Apertium hardware from Ops.
      • Apertium instance is ready to go with Service Cluster A as per Alex/Kartik's estimation of CPU/RAM monitoring for Beta.
  • Varnish setup on Beta (and Production).
    • Started: 2014-11-19
    • Ticket: https://phabricator.wikimedia.org/T76200
    • Status:
      • Assigned to Alex.
      • deployment-sca-cache01.eqiad.wmflabs instance has been created (2014-11-25) Yes Done
      • Alex need to setup Virtual IP/instance to finalize VCL rules for Varnish on Beta.
  • Security review of Content Translation "Beta Feature"

In Progress

[edit]
  • Testing!

To be started

[edit]
  • None

Stalled

[edit]
  • None
Retrieved from "https://www.mediawiki.org/w/index.php?title=Content_translation/Deployments/Deployment_Plan/Archive4&oldid=3605650"