Definition
Term | Definition |
Big keys: | l Keys that have a large value. If the size of a single String key exceeds 10 KB, or if the size of all elements of a key combined exceeds 1 MB, the key is defined as a big key. l Keys that have a large number of elements. If the number of elements in a key exceeds 3,000, the key is defined as a big key. |
Hot keys: | l A key is defined as a hot key if it is frequently requested or if it occupies a large number of resources. For example: l A key that receives a large number of queries per second (QPS) is considered a hot key. For example, if an instance has a total QPS of 5,000 and one key in the instance receives 3,000 QPS, the key is considered a hot key. l A key that has a high bandwidth usage is considered a hot key. For example, if a key 1 MB in size receives a large number of requests per second and occupies a large bandwidth, the key is considered a hot key. |
Issues Caused by Big Keys and Hot Keys
Scenario | Issue Description |
Big keys: | It takes longer for the client to run commands. |
Operations may be blocked, important keys may be evicted, or out-of-memory (OOM) errors may even occur when the memory usage of Redis reaches the upper limit specified by the maxmemory parameter. | |
When read requests are frequently made for a big key, a large amount of network bandwidth is occupied and the response time may increase and other services may be affected. | |
The primary database may be blocked for an extended period of time while a big key is being modified or deleted. This may lead to a synchronization failure or an active/standby switchover. | |
Memory usage may be uneven for a cluster instance. | |
Hot keys: | Hot keys cause high CPU usage, which slows down the response of other requests and degrades the performance of instances. |
One data shard in an instance receives a large number of requests while other data shards in the instance remain idle in a cluster instance. A single shard becomes a performance bottleneck. | |
A cache breakdown occurs if a hot key receives more requests than that can be handled by the cache. In this case, a large number of requests are directly sent to the backend database and may lead to a database breakdown, seriously affecting business. |
Causes for Big Keys and Hot Keys
Big keys and hot keys may occur for a variety of reasons, such as incorrect use of Redis, insufficient workload planning, accumulation of invalid data, and traffic spikes.
Causes for Big Keys:
l Incorrect use of Redis: If Redis is used in an improper scenario, the size of a key may be larger than necessary. For example, if a string key is used to store a binary file that is large in size, the size of the key may be larger than necessary.
l Insufficient workload planning: members are not split between keys properly and some keys have more members than required.
l Accumulation of invalid data: Invalid data is not deleted on a regular basis. For example, the number of members for a list/hash key constantly increases because invalid data is not timely cleared.
Causes for Hot Keys:
l Improper business arrangement: Keys are not properly split, and service requests are sent to one key.
l Unexpected traffic spikes: Unexpected traffic spikes may occur for a variety of reasons, such as hot news.
Identifying Bit Keys and Hot Keys
Methods | Description |
Function analysis | |
Identify big keys and hot keys by using the bigkeys and hotkeys parameters in redis-cli. | The bigkeys parameter provided by Redis allows redis-cli to traverse all keys in a Redis instance and return the statistics of keys and the big keys in Top1 of each data type. The bigkeys parameter can return statistics for keys of six data types: STRING, LIST, HASH, SET, ZSET and STREAM. Sample command: redis-cli -h <connection address of the instance> -p <port> -a <password> --bigkeys. |
Since Redis 4.0, the hotkeys parameter is provided to help you quickly identify hot keys. Sample command: redis-cli -h <connection address of the instance> -p <port> -a <password> --hotkeys. You can obtain details of hot keys in the summary of results. | |
Analyze a specified key by using built-in commands of Redis. | The following section lists low-risk commands for analyzing keys of various data types to determine whether a key is a big key.
l For a string key, you can run the STRLEN command. This command returns the length (number of bytes) of a string value stored in the key. l For a list key, you can run the LLEN command. This command returns the length of a list value stored in the key. l For a hash key, you can run the HLEN command. This command returns the number of members in the key. l For a set key, you can run the SCARD command. This command returns the number of members in the key. l For a ZSET key, you can run the ZCARD command. This command returns the number of members in the key. l For a stream key, you can run the XLEN command. This command returns the number of members in the key. |
Identify hot keys at the business layer | This method allows you to add code to the business layer to record requests that were sent to Redis instances and asynchronously analyze the collected statistics. |
Optimizing Big Keys and Hot Keys
Scenario | Methods | Description |
Big keys: | Split big keys. | For example, you can split a hash key that contains tens of thousands of members into multiple hash keys that have proper numbers of members. For Redis cluster instances, you can split big keys to balance the memory usage across multiple data shards. |
Delete big keys. | You can store data that is not suitable for Redis in other storage and delete the data from Redis. | |
Delete expired data on a regular basis. | The accumulation of expired data leads to big keys. For example, if you incrementally write a large amount of data to list or hash data and ignore the timeliness of the data, big keys may be generated. You can use scheduled tasks to delete expired data. You can use it with the cache analysis function. | |
Hot keys: | Use the local cache on the client. | You can design a two-level cache architecture (client/local cache and Redis). Hotspot data is obtained from the local cache first. Data that is frequently modified is preferentially cached locally. |
Replicate hot keys for Redis instances. | In a Redis cluster, you can replicate the hot key in a data shard to generate new hot keys and migrate the new hot keys to other data shards. For example, you can replicate a hot key in a data shard to generate three identical hot keys. Then, you can migrate the hot keys to other data shards to reduce the pressure on the single data shard. |