Importance of Retry

Redis client retry is an important mechanism to tackle network failures, heavy server load, and command execution failures, thereby improving system reliability and stability. You can configure the maximum number of retries, control the interval between retries, and use the exponential backoff policy to effectively deal with the failures of Redis operations. You can also configure Jedis connection pool parameters properly in a way to fully utilize connection resources and improve connection availability. Examples of parameters: maximum number of connections, and maximum number of idle connections. The retry mechanism reduces the impact of Redis operation failures and ensures the normal running of the system.

Scenario

Scenario	Description
Network failures or interruptions	Communication with the Redis server fails or is unstable due to network connection interruptions or instability.
An active/standby switchover triggered by a fault	If the active node is faulty due to Redis underlying hardware or other reasons, an active/standby switchover is triggered to ensure that the instance is still available. An active/standby switchover has the following impacts:Second-level disconnectionRead-only for up to 30s
Heavy load on the Redis server	The Redis server is overloaded and cannot respond to requests promptly.
Failure to execute Redis commands	Redis commands fail to be executed possibly because a write conflict occurs or the resources of the Redis server are exhausted.
Request blockage caused by slow queries	Operations whose time complexity is O(N) cause slow queries and request blockage. In this case, other client requests may temporarily fail.
Reboot or upgrade of the Redis server	The Redis server is being rebooted or upgraded and cannot process requests properly.
Incorrect Redis configurations	The operation fails because Redis is incorrectly configured or incompatible commands are used.

Business Requirements

The connection needs to be automatically retried in case of a connection error, timeout or other faults.
The number and interval of retries must be controlled to prevent excessive retries from affecting system performance.

Analysis

Retry times and interval: Determine the number of retries and the interval for retries to meet the requirements of business and system load.
Detect exceptions: Detect connection exceptions and timeout in the code block used for communicating with the Redis server.
Retry: Retry after an exception is detected based on the configured retry times and intervals.
Number of retries: When the number of retries reaches the configured limit, stop retry and handle the final results of a connection error or timeout.

Recommended Retry Rules

Retry Rules	Description
Set the maximum number of retries.	Limit the number of retries to prevent the program from long blockage due to unlimited retries.
Control the retry interval.	Set the retry interval properly to prevent excessive pressure on the Redis server due to frequent retries.
Use the exponential backoff policy.	Gradually increase the interval between retries to avoid performance bottlenecks caused by massive concurrent requests.
Set an upper limit for the number of retries.	Set an upper limit for the number of retries to prevent excessive retries from exhausting system resources.
Record retry exceptions and print failure reports.	During a retry, you can print retry error logs at the WARN level.

Implementation Solution

Retries are not supported in JedisPool. You can implement retries by encapsulate the retry code for JedisPool.

Recommended Parameter Settings for Jedis Connection Pools

Parameter	Description	Recommended Setting
maxTotal	Maximum number of connections in a connection pool	Set this parameter based on actual conditions to avoid wasting resources.
maxIdle	Maximum number of idle connections in a connection pool	Set this parameter based on actual conditions.
minIdle	Minimum number of idle connections in a connection pool	Set this parameter based on actual conditions to ensure the availability of idle connections.
maxWaitMillis	Maximum waiting time for obtaining a connection	Set this parameter based on actual conditions to avoid timeout due to waiting for connections.
testOnBorrow	Indicates whether to check the connection validity when obtaining connections from the connection pool.	You are advised to set this parameter toTrue .

The following code example uses Java and a Jedis client library for retries:

import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.JedisPoolConfig;

public class RedisClient {
    private static final String REDIS_HOST = "localhost";
    private static final int REDIS_PORT = 6379;
    private static final int MAX_TOTAL_CONNECTIONS = 10;
    private static final int MAX_IDLE_CONNECTIONS = 5;
    private static final int MIN_IDLE_CONNECTIONS = 1;
    private static final long MAX_WAIT_MILLIS = 5000;
    private static final boolean TEST_ON_BORROW = true;
    private static final int MAX_RETRIES = 3;

    private JedisPool jedisPool;

    public RedisClient() {
        JedisPoolConfig poolConfig = new JedisPoolConfig();
        poolConfig.setMaxTotal(MAX_TOTAL_CONNECTIONS);
        poolConfig.setMaxIdle(MAX_IDLE_CONNECTIONS);
        poolConfig.setMinIdle(MIN_IDLE_CONNECTIONS);
        poolConfig.setMaxWaitMillis(MAX_WAIT_MILLIS);
        poolConfig.setTestOnBorrow(TEST_ON_BORROW);

        jedisPool = new JedisPool(poolConfig, REDIS_HOST, REDIS_PORT);
    }

    public void performRedisOperation() {
        Jedis jedis = null;
        int retryCount = 0;
      
        while (retryCount < MAX_RETRIES) {
            try {
                jedis = jedisPool.getResource();
                // Perform Redis operations
                // jedis.set("key", "value");
                // ...
                break;  // The command is executed successfully. Exit from the retry loop.
            } catch (Exception e) {
                //An exception occurs. Perform a retry.
                retryCount++;

In this example, a Jedis connection pool is created and configured using JedisPoolConfig. You can configure the above parameters to improve the performance of the connection pool.

During Redis operations, a retry mechanism is used when an exception occurs. You can adjust the retry policy by changing the maximum number of retries.

The example code is for reference only, and can be adjusted and optimized according to your requirements. The Redis client retry mechanism improves system stability and fault tolerance. You need to configure the retry times, interval, and exception handling logic properly to cater to service requirements.

DCS Redis Help Documentation

Redis Client Retry