Redis: Difference between revisions
→Setup: php5-redis no longer exists |
update urls |
||
Line 52: | Line 52: | ||
===General=== |
===General=== |
||
* [ |
* [https://redis.io/ Official site] (see esp. [https://redis.io/topics/introduction Introduction to Redis]) |
||
* The [[:w:Redis|Redis]] article on the English Wikipedia. |
* The [[:w:Redis|Redis]] article on the English Wikipedia. |
||
* [ |
* [https://redislabs.com/resources/redis-watch-archive/ Redis Watch] - an e-mail round-up of Redis news, articles, tools and libraries |
||
* [[Redis/INCR]] |
* [[Redis/INCR]] |
||
* [ |
* [https://livebook.manning.com/#!/book/redis-in-action/chapter-1/1 Getting to Know Redis] |
||
* [ |
* [https://blog.mjrusso.com/2010/10/17/redis-from-the-ground-up.html Redis, from the Ground Up] |
||
* [http://liamkaufman.com/blog/2012/06/04/redis-and-relational-data/ Redis and Relational Data] |
* [http://liamkaufman.com/blog/2012/06/04/redis-and-relational-data/ Redis and Relational Data] |
||
* [ |
* [https://www.amazon.com/Redis-Cookbook-Tiago-Macedo/dp/1449305040 Redis Cookbook ] (book; not great, but see ch. "Analytics and Time-Based Data") |
||
* [ |
* [https://changelog.com/podcast/45 Interview with Salvatore Sanfilippo] (code-oriented but still useful) |
||
* [https://groups.google.com/group/redis-db Redis DB] (Google Group) |
* [https://groups.google.com/group/redis-db Redis DB] (Google Group) |
||
Line 66: | Line 66: | ||
* [http://bretthoerner.com/2011/2/21/redis-at-disqus/ Redis at Disqus] (their entire analytics platform runs on Redis) |
* [http://bretthoerner.com/2011/2/21/redis-at-disqus/ Redis at Disqus] (their entire analytics platform runs on Redis) |
||
* [http://filer.progstr.com/1/post/2012/03/effective-web-app-analytics-with-redis.html Effective Web App Analytics with Redis] |
* [http://filer.progstr.com/1/post/2012/03/effective-web-app-analytics-with-redis.html Effective Web App Analytics with Redis] |
||
* [ |
* [https://www.youtube.com/watch?v=RlkCdM_f3p4 How YouPorn uses Redis] (video) |
||
* [http://blog.getspool.com/2011/11/29/fast-easy-realtime-metrics-using-redis-bitmaps/ Realtime metrics using Redis bitmaps] |
* [http://blog.getspool.com/2011/11/29/fast-easy-realtime-metrics-using-redis-bitmaps/ Realtime metrics using Redis bitmaps] |
||
Line 73: | Line 73: | ||
* [https://github.com/andymccurdy/redis-py/ redis-py] is the library of choice for Python |
* [https://github.com/andymccurdy/redis-py/ redis-py] is the library of choice for Python |
||
* [http://degizmo.com/2010/03/22/getting-started-redis-and-python/ Getting Started: Redis and Python] |
* [http://degizmo.com/2010/03/22/getting-started-redis-and-python/ Getting Started: Redis and Python] |
||
* [ |
* [https://www.slideshare.net/josiahcarlson/python-redis-talk-13721977 Redis and Python] (presentation slides) |
||
* [https://github.com/blog/542-introducing-resque Resque] for jobs |
* [https://github.com/blog/542-introducing-resque Resque] for jobs |
||
* [https://github.com/iamteem/redisco/ Redisco], a Python ORM for Redis |
* [https://github.com/iamteem/redisco/ Redisco], a Python ORM for Redis |
||
* [ |
* [https://pypi.org/project/analytics/ py-analytics] (I haven't used this) |
||
* [ |
* [https://github.com/bilus/redis-bitops redis-bitops] Ruby gem for sparse bitmap operations |
||
===Informed Opinions=== |
===Informed Opinions=== |
||
Line 83: | Line 83: | ||
===Miscellaneous=== |
===Miscellaneous=== |
||
* [ |
* [https://engineering.instagram.com/storing-hundreds-of-millions-of-simple-key-value-pairs-in-redis-1091ae80f74c Storing hundreds of millions of simple key-value pairs] (how Instagram uses Redis) |
||
* [https://www.datadoghq.com/blog/how-to-monitor-redis-performance-metrics/?ref=wikipedia Key performance metrics to monitor for Redis] |
* [https://www.datadoghq.com/blog/how-to-monitor-redis-performance-metrics/?ref=wikipedia Key performance metrics to monitor for Redis] |
Revision as of 19:31, 4 April 2018
Redis is an open-source, networked, in-memory, key-value data store with optional durability, written in ANSI C.
Setup
If you haven't already, you'll need to configure a Redis instance and install a Redis client library for PHP. Most environments require the phpredis PHP extension. On Debian / Ubuntu, you can install the requirements via apt-get install redis-server php-redis
. If you're running MediaWiki on top of HHVM (unlikely unless you're a developer), you can use the Redis client library that comes with HHVM instead.
In your LocalSettings.php file, set:
/** @see RedisBagOStuff for a full explanation of these options. **/
$wgObjectCaches['redis'] = array(
'class' => 'RedisBagOStuff',
'servers' => array( '127.0.0.1:6379' ),
// 'connectTimeout' => 1,
// 'persistent' => false,
// 'password' => 'secret',
// 'automaticFailOver' => true,
);
You'll now be able to acquire a Redis object cache object via wfGetCache( 'redis' )
. If you'd like to use Redis as the default cache for various data, you may set any of the following configuration options:
$wgMainCacheType = 'redis';
$wgSessionCacheType = 'redis'; // same as WMF prod
// Not widely tested:
$wgMessageCacheType = 'redis';
$wgParserCacheType = 'redis';
$wgLanguageConverterCacheType = 'redis';
Job queue
$wgJobTypeConf['default'] = [
'class' => 'JobQueueRedis',
'redisServer' => '127.0.0.1:6379',
'redisConfig' => [],
'claimTTL' => 3600,
'daemonized' => true
];
From that moment, jobs will be delivered to the Redis instance run the specified server.
MediaWiki & Wikimedia use cases for Redis
- Session storage
- The Wikimedia Foundation has been using Redis as a memcached replacement for session storage since the eqiad switchover in January 2013, because it has a replication feature which can be used to synchronise data between the two data centres. It allowed us to switch from Tampa to Ashburn without logging everyone out.
- Job queue
- We previously stored the MW job queue in MySQL. This gave us lots of useful features, like replication and indexing for duplicate removal, but it has often been hard to manage the performance implications of the high insert rate. Among its many features, Redis embeds a Lua interpreter on the server side. The new Redis job queue class provides a rich feature set superior to the MySQL job queue, mainly through several server-side Lua scripts which provide high-level job queue functions. Redis is also used to keep a hash table that tracks which job queues actually have jobs, so runners know where to look. Updates to this table are push-based, so it is always up-to-date.
- Features
- Extension:GettingStarted's early implementation of a category-based recommender system has used Redis to store a list of tasks (actually page ids) served via a few interfaces.
Further reading
General
- Official site (see esp. Introduction to Redis)
- The Redis article on the English Wikipedia.
- Redis Watch - an e-mail round-up of Redis news, articles, tools and libraries
- Redis/INCR
- Getting to Know Redis
- Redis, from the Ground Up
- Redis and Relational Data
- Redis Cookbook (book; not great, but see ch. "Analytics and Time-Based Data")
- Interview with Salvatore Sanfilippo (code-oriented but still useful)
- Redis DB (Google Group)
Analytics
- Redis at Disqus (their entire analytics platform runs on Redis)
- Effective Web App Analytics with Redis
- How YouPorn uses Redis (video)
- Realtime metrics using Redis bitmaps
Tooling
- Redsmin a real-time, atomic, performant administration and monitoring interface for Redis
- redis-py is the library of choice for Python
- Getting Started: Redis and Python
- Redis and Python (presentation slides)
- Resque for jobs
- Redisco, a Python ORM for Redis
- py-analytics (I haven't used this)
- redis-bitops Ruby gem for sparse bitmap operations
Informed Opinions
Miscellaneous
- Storing hundreds of millions of simple key-value pairs (how Instagram uses Redis)
- Key performance metrics to monitor for Redis