Redis is an open-source in-memory key-value data store. It can be used as a database, cache, and message broker. It supports various data structures such as strings, hashes, lists, sets, and sorted sets, with atomic operations for manipulating them. Redis also supports simple, powerful commands for working with data, such as “get”, “set”, “incr”, and “lpush”, among others.
Multiple Redis servers can be deployed to perform data replication. The deployment supports a master Redis server which performs read/write operations and replica servers which perform synchronization of data from the master. The replica servers initially perform full data synchronizations and then perform incremental updates. In case the replica is not able to perform partial sync the master provides a snapshot of all its data to the replica and after that incremental sync continues.
Redis has an additional layer of Sentinel which can be used to configure High Availability of multiple Redis servers and perform fail-over in case of outage of the master Redis server.
Redis Installation
The following guide lists down the procedure for installation of Redis on Ubuntu 22.04 server with Redis 7.0.7.
Install Ubuntu 22.04 with swap enabled and equal to the amount of RAM on the system.
Set the Linux kernel overcommit memory setting to 1. Add vm.overcommit_memory = 1 to /etc/sysctl.conf and run the command sysctl vm.overcommit_memory=1 to activate the setting.
Ensure the Linux kernel feature Transparent Huge Pages does not impact Redis memory usage and latency, use this command:
$ echo never > /sys/kernel/mm/transparent_hugepage/enabledAdd the package signing key and apt repository of redis.
$ curl -fsSL https://packages.redis.io/gpg | \
sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg
$ echo "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] \
https://packages.redis.io/deb $(lsb_release -cs) main" | \
sudo tee /etc/apt/sources.list.d/redis.listInstall the redis and sentinel packages
$ sudo apt update
$ sudo apt install redis redis-sentinelThe default configuration path of redis installation is /etc/redis/redis.conf. The configuration of sentinel is /etc/redis/sentinel.conf. The default logs location is /var/log/redis/redis-sentinel.log.
Enable the redis service and start the daemon.
$ sudo systemctl enable redis-server
$ sudo systemctl start redis-serverRedis HA Setup with Sentinel
The High Availability setup requires minimum of three nodes. We will deploy three redis servers with sentinel on the same three nodes. The three redis nodes will be deployed as master and replicas.
Authentication ACL
With Redis version 6+ we can configure user logins with proper ACL to manage what privileges are assigned to the authentication user. The logins, passwords and ACLs can be configured within the main redis.conf file or these can be used in a separate configuration file which can be included into the redis configuration for better management of user accounts. Create the users ACL file /etc/redis/users.acl. We require one account which will be used for the redis servers to authenticate with each other for replication. We can create account for application access with only the required privileges. Following is a replication user “replica-user” which is enabled “on”, have admin and connection privileges and have “Super-Secure-Password”.
# Replication User
user replica-user on +@admin +@connection ~* >SUPER-SECURE-PASSWORD
# Application User
user appuser on +@keyspace +@connection +set +get ~* >SECURE-APP-PASSMaster Redis Configuration
Make the following changes in /etc/redis/redis.conf on the master redis server. The file is self documented with explanations of the configuration options.
.
.
# Comment the option to listen on all interfaces
# bind 127.0.0.1 -::1
.
.
# Allow connection to redis server other than local host
protected-mode no
.
.
# Redis default listen port
port 6379
.
.
# User for replication
masteruser "replica-user"
.
.
# Priority of the mode when failing over. Lower no has higher priority
replica-priority 100
.
.
# Using an external ACL file
aclfile "/etc/redis/users.acl"
.
.Replica Redis Configuration
Make the following changes in /etc/redis/redis.conf on the replica redis servers.
.
.
# Comment the option to listen on all interfaces
# bind 127.0.0.1 -::1
.
.
# Allow connection to redis server other than local host
protected-mode no
.
.
# Redis default listen port
port 6379
.
.
# User for replication
masteruser "replica-user"
.
.
# Priority of the mode when failing over. Lower no has higher priority
replica-priority 200
.
.
# Using an external ACL file
aclfile "/etc/redis/users.acl"
.
.
# Setup replication from master
replicaof 10.54.101.34 6379
.
.Verify Replication
Restart all the redis daemons on all nodes to make sure the configurations are applied. Verify the service is started in all nodes.
$ sudo systemctl restart redis-server
$ sudo systemctl status redis-server
● redis-server.service - Advanced key-value store
Loaded: loaded (/lib/systemd/system/redis-server.service; enabled; vendor
Active: active (running) since Fri 2023-01-13 09:33:12 PKT; 8min ago
Docs: http://redis.io/documentation,
man:redis-server(1)
Main PID: 206 (redis-server)
Status: "MASTER <-> REPLICA sync: Partial Resynchronization accepted.
Tasks: 5 (limit: 18885)
Memory: 3.2M
CPU: 1.564s
CGroup: /system.slice/redis-server.service
└─206 "/usr/bin/redis-server *:6379" "" "" "" "" "" "" "" "" "" "" ""
Jan 13 09:33:12 redis1 systemd[1]: Starting Advanced key-value store...
Jan 13 09:33:12 redis1 systemd[1]: Started Advanced key-value store.Login to redis cli using the replica-user and verify replication information. The password is defined in the users.acl file.
$ redis-cli --user sentry-user --askpass
Please input password: ************************
127.0.0.1:6379> INFO replication
# Replication
role:slave
master_host:10.54.101.34
master_port:6379
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_read_repl_offset:7418683
slave_repl_offset:7418683
slave_priority:100
slave_read_only:1
replica_announced:1
connected_slaves:0
min_slaves_good_slaves:0
master_failover_state:no-failover
master_replid:bb79b6380d8e8ae9328900123636d8a704743eb7
master_replid2:8003ba6e93e7ee6d67c7c6881f8418b24d2377fd
master_repl_offset:7418683
second_repl_offset:7384378
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:7384378
repl_backlog_histlen:34306Redis HA with Sentinel
The sentinel requires user access with proper privileges to monitor the redis servers. We also need a user for sentinels to authenticate with each other.
Add a user “sentry-user” in the /etc/redis/users.acl file created for redis servers which is used for redis server monitoring.
# Replication User
user replica-user on +@admin +@connection +info ~* >SUPER-SECURE-PASSWORD
# Monitoring User for sentinel
user sentry-user on +client +subscribe +publish +ping +info +multi +slaveof +config +client +exec ~* >SECURE-PASS-FOR-SENTINELS
# Application User
user appuser on +@keyspace +@connection +set +get ~* >SECURE-APP-PASSThe sentinel does not support all the acl options as redis servers. So we will create a separate users acl file for sentinel /etc/redis/sentinel-users.acl. A user “sentry” will be used for sentinels to authenticate with each other. Second user should be the same “sentry-user” with same credentials defined in the redis users.acl file.
# Sentinel authentication
user sentry +@admin +@connection ~* on >SENTINEL-ADMIN-PASS
# User for redis monitoring
user sentry-user on +@admin +@connection ~* >SECURE-PASS-FOR-SENTINELSEdit the sentinel configuration /etc/redis/sentinel.conf with following options.
.
.
# Uncomment the following line to expose Sentinel on external interfaces:
protected-mode no
.
.
# The port that this sentinel instance will run on
port 26379
.
.
# Tells Sentinel to monitor this master
sentinel monitor redis1 10.54.101.34 6379 2
.
.
# Sentinel user to monitor redis server
sentinel auth-user redis1 sentry-user
.
.
# Number of milliseconds master should be unreachable in order to consider it DOWN
sentinel down-after-milliseconds redis1 10000
.
.
# Using an external ACL file
aclfile "/etc/redis/sentinel-users.acl"
.
.
# Sentinel to authenticate with other Sentinels with specific user name.
sentinel sentinel-user sentry
.
.
# Specifies the failover timeout in milliseconds.
sentinel failover-timeout redis1 60000
.
.Start the sentinel daemon on all sentinel nodes. Make sure the service is started.
$ sudo systemctl start redis-sentinel
$ sudo systemctl status redis-sentinel
● redis-sentinel.service - Advanced key-value store
Loaded: loaded (/lib/systemd/system/redis-sentinel.service; enabled; vendor
Active: active (running) since Fri 2023-01-13 09:33:12 PKT; 37min ago
Docs: http://redis.io/documentation,
man:redis-sentinel(1)
Main PID: 205 (redis-sentinel)
Status: "Ready to accept connections"
Tasks: 5 (limit: 18885)
Memory: 6.0M
CPU: 8.500s
CGroup: /system.slice/redis-sentinel.service
└─205 "/usr/bin/redis-sentinel *:26379 [sentinel]" "" "" "" "" "" ""
Jan 13 09:33:12 redis1 systemd[1]: Starting Advanced key-value store...
Jan 13 09:33:12 redis1 systemd[1]: Started Advanced key-value store.Verify Sentinel Monitoring
Login to sentinel CLI on one of the redis servers and verify the master redis server. We need to define the port (26379) on which the sentinel application is listening.
$ redis-cli --user sentry --askpass -p 26379
Please input password: ************************
127.0.0.1:26379> SENTINEL GET-MASTER-ADDR-BY-NAME redis1
1) "10.54.101.34"
2) "6379"Check all the other sentinels information using the same CLI.
127.0.0.1:26379> SENTINEL SENTINELS redis1
1) 1) "name"
2) "4ec8c9e1453bc8800a89169e8ca10882aa944391"
3) "ip"
4) "10.54.101.34"
5) "port"
6) "26379"
.
.
2) 1) "name"
2) "a4c53c626a09b020503427ddeeec084b1ba58d5a"
3) "ip"
4) "10.54.101.122"
5) "port"
6) "26379"
.
.When all redis servers and sentinels are properly started and can see each other, the sentinel process rewrites its configuration with the information of all other redis and sentinel nodes.
The configuration file on redis master at the end should be as follows:
.
.
# Generated by CONFIG REWRITE
latency-tracking-info-percentiles 50 99 99.9
sentinel myid 26f12ac6a2f601855061899c0ce80fee8ee7cffc
sentinel config-epoch redis1 3
sentinel leader-epoch redis1 3
sentinel current-epoch 3
sentinel known-replica redis1 10.54.101.122 6379
sentinel known-replica redis1 10.54.101.23 6379
sentinel known-sentinel redis1 10.54.101.122 26379 a4c53c626a09b020503427ddeeec084
sentinel known-sentinel redis1 10.54.101.34 26379 4ec8c9e1453bc8800a89169e8ca10882Check the log files of redis and sentinel to verify service operations.
$ sudo tail -f /var/log/redis/redis-server.log
206:S 13 Jan 2023 09:33:15.972 * MASTER <-> REPLICA sync started
206:S 13 Jan 2023 09:33:16.975 * Connecting to MASTER 10.54.101.34:6379
206:S 13 Jan 2023 09:33:16.976 * MASTER <-> REPLICA sync started
206:S 13 Jan 2023 09:33:16.976 * Non blocking connect for SYNC fired the event.
206:S 13 Jan 2023 09:33:16.976 * Master replied to PING, replication can continue...
206:S 13 Jan 2023 09:33:16.976 * Trying a partial resynchronization (request 8003ba6e93e7ee6d67c7c6881f8418b24d2377fd:7384378).
206:S 13 Jan 2023 09:33:16.976 * Successful partial resynchronization with master.
206:S 13 Jan 2023 09:33:16.976 # Master replication ID changed to bb79b6380d8e8ae9328900123636d8a704743eb7
206:S 13 Jan 2023 09:33:16.976 * MASTER <-> REPLICA sync: Master accepted a Partial Resynchronization.$ sudo tail -f /var/log/redis/redis-sentinel.log
205:X 13 Jan 2023 09:33:12.876 * Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.
205:X 13 Jan 2023 09:33:12.876 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
205:X 13 Jan 2023 09:33:12.876 # Redis version=7.0.7, bits=64, commit=00000000, modified=0, pid=205, just started
205:X 13 Jan 2023 09:33:12.876 # Configuration loaded
205:X 13 Jan 2023 09:33:12.877 * monotonic clock: POSIX clock_gettime
205:X 13 Jan 2023 09:33:12.878 * Running mode=sentinel, port=26379.
205:X 13 Jan 2023 09:33:12.880 # Sentinel ID is 26f12ac6a2f601855061899c0ce80fee8ee7cffc
205:X 13 Jan 2023 09:33:12.880 # +monitor master redis1 10.54.101.34 6379 quorum 2References
https://redis.io/docs/getting-started/installation/install-redis-on-linux/
https://redis.io/docs/management/replication/
https://redis.io/docs/management/sentinel/
https://redis.io/docs/manual/cli/