Wikimedia Developer Summit/2017/Integrating MediaWiki (and other services) with dynamic configuration
Session Overview
[edit]Title: Integrating MediaWiki (and other services) with dynamic configuration
Day & Time: Tuesday Jan. 10th, 11:00 am – 12:10 pm (19:00 – 20:10 UTC)
Room: Hawthorne
Phabricator Task Link: https://phabricator.wikimedia.org/T149617
Facilitator(s): paravoid, greg-g
Note-Taker(s): Volans, bd808
Remote Moderator:
Advocate:
Session Summary
[edit]
Purpose
[edit]Find a unified, simple and reliable way to manage the dynamic configuration (live state) of services in WMF.
Agenda
[edit]40 minutes presentation, 30 minutes discussions and conclusions
Style
[edit]Consensus: coming to agreement on one solution
Discussion Topics
[edit]The current way of managing the state of a cluster in WMF is via it's configuration and each change requires a change in the configuration repository (Puppet).
The live state of a service or cluster should be managed separately in a simple and reliable way that allows to change the state quickly, reliably and without running Puppet.
The main two categories of live state are:
- Service discovery (SOA address book)
- State management
Two solutions were proposed, based on the different use case:
- DNS for simple data structures and service discovery through TXT or URI records
- Confd for more complex data structure or integrating an etcd client in the application layer itself
For the full session notes see the detailed notes.
Action Items
- Stop relying on a configuration repository to store the live state of clusters, it needs to be treated separately.
- Use DNS for service location/discovery, and simple data structures, but has some limitations.
- Use Confd with templates and scripts to manage more complex data structures.
- For the MediaWiki specific case, there are a bunch of options:
- generate a JSON file that is read by the application, parsed by MediaWiki and cached into APC.
- generate a PHP file that will be included directly by the application.
- Add safe measures to ensure that the configuration is consistent across the cluster and no hosts have a stale one.