Redis is an in-memory data structure store that's written entirely in C. It's able to easily perform millions of operations a second with sub millisecond latency. Redis can achieve these speeds by storing its dataset in memory, as opposed to traditional SQL databases which write their data to disk.
Redis has been in development since 2009 and is currently being used by some of the largest companies in the world, such as Twitter, GitHub, StackOverflow, Flickr, Snapchat and Airbnb.
It's commonly used for caching, queues, session storage, very large datasets, full text search and time series data.
Some Redis use cases include realtime web analytics, ecommerce shopping cart session storage, chat/messaging systems, gaming leaderboards and more.
Redis has no dependencies other than the GCC compiler and libc:
sudo apt-get update && sudo apt-get upgrade sudo apt-get install gcc libc6-dev
If you're going to run the optional make test command during installation, you'll also need the tcl package:
sudo apt-get install tcl
Download and extract the latest version of Redis:
curl -o /tmp/redis-stable.tar.gz http://download.redis.io/redis-stable.tar.gz tar xzvf /tmp/redis-stable.tar.gz
Compile and install Redis:
cd /tmp/redis-stable make
Check that the build works correctly (optional):
Install Redis to system:
sudo make install
Before running Redis for the first time, you'll need to create a configuration file and set the permissions for it.
Add Redis user and group:
sudo adduser --system --group --no-create-home --gecos 'Redis Server' redis
Create Redis config directory:
sudo mkdir /etc/redis
Copy redis.conf over from tmp directory:
sudo cp /tmp/redis-stable/redis.conf /etc/redis/
Set ownership of /etc/redis to redis user:
sudo chown -R redis:redis /etc/redis
Prevent other users from being able to read redis.conf:
sudo chmod 600 /etc/redis/redis.conf
Redis was not designed with security in mind. It was designed to be accessed inside of a trusted environment that does not directly face the internet. By default Redis does not have a password set.
Redis is extremely fast. A malicious user can attempt up to 150,000 password combinations a second.
You can either generate your own secure password, or use this Bash one-liner to generate a 64 character password for you:
echo "yourpassword" | sha256sum | base64 | head -c 64 ; echo
This will create a SHA-256 checksum of your password, Base64 encode it and then trim it to 64 characters in length. You'll see an output like:
Next you'll need to put your password into Redis' config file:
# /etc/redis/redis.conf requirepass NGQ3YTViYmQ1MDFmMTlkZWYxNTI0MTE1M2RmZjRmNTUxMmNlODA4ODA1MDViMTUw
Logging is turned off by default.
To enable logging with Redis specify a filename for the logfile option:
# /etc/redis/redis.conf logfile /var/log/redis.log
Then create the file and set permissions for it:
sudo touch /var/log/redis.log sudo chown redis:redis /var/log/redis.log
Redis has two persistence options:
- Redis Database File (RDB): this creates snapshots of your data at different time intervals.
- Append Only File (AOF): this will log every write operation received by the server and construct the original dataset each time the server starts.
You can use either persistence option, or a combination of both.
It's worth reviewing the Redis persistence documentation to understand the key advantages and disadvantages of each option.
If you're not planning on having data persistence then set the value of save to "" and comment out any other occurences of it:
# /etc/redis/redis.conf # save 900 1 # save 300 10 # save 60 10000 save ""
If you want data persistence you'll need to specify the data directory you want to use:
# /etc/redis/redis.conf dir /var/lib/redis
Then create the data directory and set the permissions for it:
sudo mkdir /var/lib/redis sudo chown redis:redis /var/lib/redis sudo chmod 770 /var/lib/redis
By default Redis will create snapshots (RDB) of your data to disk (/var/lib/redis/dump.rdb) when the following conditions are met:
- Every 15 minutes if at least 1 key changed
- Every 5 minutes if at least 10 keys changed
- Every minute if at least 10000 keys changed
To change the default RDB options, you can set the save configuration option accordingly:
# /etc/redis/redis.conf save <seconds> <changes>
The problem with snapshotting (RDB) is that if you accidentally kill the Redis server process, if the system's power goes out, or if the system crashes, you'll lose the latest data.
This is the key advantage of AOF persistence as the data is written to disk.
To enable AOF persistence set appendonly to yes:
# /etc/redis/redis.conf appendonly yes
Then choose from one of the following options for how often the operations should be appended to file:
- no: don't fsync, just let the OS flush the data when it wants. Faster.
- always: fsync after every write to the append only log. Slow, Safest.
- everysec: fsync only one time every second. Compromise.
# /etc/redis/redis.conf appendfsync everysec
Limit memory usage
You'll need to set a maximum memory usage limit for Redis. If there's no limit set, Redis will continue to store more and more data, potentially consuming more memory than is available on your system. This can lead to nasty unexpected behavior and even cause your server to crash.
Set the maximum memory usage with the maxmemory option:
# /etc/redis/redis.conf maxmemory 500mb
You'll also need to instruct Redis what to do when it has reached the maximum memory usage limit:
# /etc/redis/redis.conf maxmemory-policy noeviction
There's a few different policy options available:
# volatile-lru -> Evict using approximated LRU among the keys with an expire set. # allkeys-lru -> Evict any key using approximated LRU. # volatile-lfu -> Evict using approximated LFU among the keys with an expire set. # allkeys-lfu -> Evict any key using approximated LFU. # volatile-random -> Remove a random key among the ones with an expire set. # allkeys-random -> Remove a random key, any key. # volatile-ttl -> Remove the key with the nearest expire time (minor TTL) # noeviction -> Don't evict anything, just return an error on write operations. # # LRU means Least Recently Used # LFU means Least Frequently Used
Run Redis as System Service
Systemd requires Redis to run non-daemonized:
# /etc/redis/redis.conf daemonize no
Have Redis run under systemd:
# /etc/redis/redis.conf supervised systemd
Create a Redis systemd service file:
# /etc/systemd/system/redis.service [Unit] Description=Redis Server After=network.target [Service] User=redis Group=redis ExecStart=/usr/local/bin/redis-server /etc/redis/redis.conf ExecStop=/usr/local/bin/redis-cli shutdown Restart=always Type=notify [Install] WantedBy=multi-user.target
Enable Redis at boot
sudo systemctl enable redis
sudo systemctl start redis
Test Redis Installation
At this point, it's a good idea to make sure Redis is functioning properly.
You can use the redis-cli tool to send and receive commands to Redis:
Once connected you'll need to authenticate yourself with the password:
127.0.0.1:6379> auth PASSWORD
Next type PING. In return you will receive a PONG response. You can use this command to verify that your connection is still alive:
127.0.0.1:6379> PING PONG
If you have data persistence enabled, make sure that the dump file is being stored properly to /var/lib/redis/dump.rdb:
Set a value to a key:
127.0.0.1:6379> set test "hello world!"
Get the value associated with that key:
127.0.0.1:6379> get test "hello world!"
Type quit to quit redis-cli:
Redis is fast. But its speed can vary depending on your server's hardware.
If you're interested in benchmarking your Redis server and seeing what type of performance it's capable of, you can use the redis-benchmark tool:
/usr/local/bin/redis-benchmark -q -n 100000 -c 50 -P 12
This will send 100,000 requests with 50 parallel connections. You'll see an output like:
[email protected]:~$ /usr/local/bin/redis-benchmark -q -n 100000 -c 50 -P 12 PING_INLINE: 740740.69 requests per second PING_BULK: 1388889.00 requests per second SET: 1315789.50 requests per second GET: 1470588.12 requests per second INCR: 1470588.12 requests per second LPUSH: 1351351.38 requests per second RPUSH: 1333333.25 requests per second LPOP: 1470588.12 requests per second RPOP: 1470588.12 requests per second SADD: 1333333.25 requests per second HSET: 1234567.88 requests per second SPOP: 1470588.12 requests per second LPUSH (needed to benchmark LRANGE): 1351351.38 requests per second LRANGE_100 (first 100 elements): 1250000.00 requests per second LRANGE_300 (first 300 elements): 1250000.00 requests per second LRANGE_500 (first 450 elements): 1250000.00 requests per second LRANGE_600 (first 600 elements): 1250000.00 requests per second MSET (10 keys): 543478.25 requests per second
Always overcommit memory
WARNING overcommit_memory is set to 0! Background save may fail under low memory condition.
Redis forks background processes in order to create RDB snapshots of your data to disk. If your system runs low on memory during this process, the Linux Out of Memory (OOM) killer will kill the fork(s) causing the data to be lost.
To avoid this you can set overcommit_memory to 1:
echo 1 > /proc/sys/vm/overcommit_memory
Set maximum number of pending connection requests
WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
SOMAXCONN specifies the maximum number of connection requests that can be queued for any given listening socket.
The value of SOMAXCONN should not exceed 65535 (USHRT_MAX).
Raise SOMAXCONN to the maximum value in order to support large bursts of requests:
sysctl -w net.core.somaxconn=65535
To make this setting persist when the server reboots, add the following line to rc.local:
# /etc/rc.local net.core.somaxconn=65535
Congratulations! At this point you should have successfully installed and configured your own Redis server. Did you have any issues with the installation? Do you think there's something useful that could be added to this guide? Let me know in the comments below.