Core Platform Team/Initiatives/Enable Multi-DC Session Storage

Enable Multi-DC Session Storage

Product Owner:	Evan Prodromou
Technical Contact:	Eric Evans
Status:	Completed
Initiative Vision:
Initiative Description:	Read Edit
Epics, User Stories, and Requirements:	Read Edit
Time and Resource Estimates:	Read Edit
Open Questions:	Read Edit
Documentation Links:	Read Edit

Initiative Description

< Initiatives

Summary

Develop a multi-master replicated key-value storage service, the semantics of which permit session access from MediaWiki in an active-active, multi-datacenter configuration. Secondarily, the service decouples MediaWiki from storage, creating additional isolation of sensitive data.

Significance and Motivation

This is a blocker to enable active-active data center. Enables multi-data center session access. Makes the system more fault tolerant and resistant. Secondarily, it isolates session data.

Outcomes

Increase the scalability of the platform for future applications and new types of content, as well as a growing user base and amount of content

Baseline Metrics

Sessions are accessed from 1 Data Center

Target Metrics

Sessions can be accessed from 2 Data Centers

Stakeholders

SRE
Performance

Known Dependencies/Blockers

Setup Kubernetes security zone (SRE)
Security review (Security - 30 day lead time)

Epics, User Stories, and Requirements

< Initiatives

Hardware request and setup
RFC for the session storage API
Investigate use of Redis session storage to see if there is extra work
Design implementation (storage, replication semantics, performance)
Test and prototype in multiple languages to understand performance/latency/throughput
Implementation
Figure out deployment method
CI for build testing docker image creation
Cassandra cluster configuration
Beta deployment
Develop migration plan
Integrate with MediaWiki
Determine if “Set if not exist” functionality is needed (implement if needed)
Determine if Per operation defined TTLs are needed (implement if needed)
Enable functional testing (set up and tear down of Cassandra)
Security review
Implementing service-checker functionality (endpoint monitoring)
Figure out the Kubernetes deployment (Helm charts)
Deploy according to migration plan (test wikis, etc…)