Cline

# Code Style and Conventions Standards for Redis This document outlines the coding style and conventions to be followed when contributing to the Redis project. Adhering to these guidelines ensures code readability, maintainability, and consistency across the codebase. These conventions are intended to be used by both human contributors and AI-powered coding assistants to maintain a high standard of quality. This document assumes familiarity with C programming, data structures, and basic Redis concepts. ## 1. Formatting Consistent formatting is crucial for readability. We primarily follow the style of the existing Redis codebase. ### 1.1. Indentation * **Do This:** Use 4 spaces for indentation. Tabs are strictly forbidden. * **Don't Do This:** Use tabs or inconsistent indentation. * **Why:** Spaces provide consistent rendering across different editors and platforms. """c // Correct indentation int my_function(int arg1, int arg2) { if (arg1 > arg2) { return arg1 - arg2; } else { return arg2 - arg1; } } // Incorrect indentation (tabs) int my_function(int arg1, int arg2) { \tif (arg1 > arg2) { \t\treturn arg1 - arg2; \t} else { \t\treturn arg2 - arg1; \t} } // Incorrect indentation (inconsistent) int my_function(int arg1, int arg2) { if (arg1 > arg2) { return arg1 - arg2; } else { return arg2 - arg1; } } """ ### 1.2. Line Length * **Do This:** Limit lines to a maximum of 80 characters. Break long lines appropriately. * **Don't Do This:** Allow lines to exceed 80 characters unless absolutely necessary (e.g., very long string literals). * **Why:** Limits improve readability on smaller screens and when viewing diffs. """c // Correct line breaking void very_long_function_name( int very_long_argument_name1, int very_long_argument_name2) { // ... code ... } // Incorrect line length void very_long_function_name(int very_long_argument_name1, int very_long_argument_name2) { // ... code ... } """ ### 1.3. Spaces * **Do This:** * Use spaces around operators (=, +, -, *, /, %, ==, !=, >, <, >=, <=, &&, ||, etc.). * Use spaces after commas in argument lists. * Use a space after "if", "for", "while", and "switch" keywords. * No space between function name and parentheses. * **Don't Do This:** Omit spaces around operators or after commas, or include spaces between function names and parentheses. * **Why:** Clear spacing enhances readability. """c // Correct use of spaces int result = a + b * (c - d); my_function(arg1, arg2, arg3); if (x > 0) { /* ... */ } // Incorrect use of spaces int result=a+b*(c-d); my_function(arg1,arg2,arg3); if(x>0){ /* ... */ } """ ### 1.4. Braces * **Do This:** Always use braces for "if", "else", "for", "while", and "do-while" statements, even for single-line bodies. Place the opening brace on the same line as the statement. * **Don't Do This:** Omit braces for single-line bodies, or place the opening brace on a new line. * **Why:** Braces prevent errors during modification and improve code consistency. """c // Correct brace usage if (condition) { do_something(); } else { do_something_else(); } for (int i = 0; i < 10; i++) { process_data(i); } // Incorrect brace usage if (condition) do_something(); //This is wrong and discouraged if(condition){ // Also wrong. do_something(); } """ ### 1.5. Vertical Spacing * **Do This:** Use blank lines to separate logical blocks of code within a function. Separate function definitions with blank lines. * **Don't Do This:** Compress code too tightly or leave excessive blank lines. * **Why:** Enhanced readability. """c // Correct Vertical Spacing int process_data(int data) { // Initialization int result = 0; // Processing logic if (data > 0) { result = data * 2; } else { result = data / 2; } return result; } int main() { int value = 10; int processed_value = process_data(value); printf("Processed value: %d\n", processed_value); return 0; } """ ## 2. Naming Conventions Clear and consistent naming is essential for understanding code. ### 2.1. General Naming * **Do This:** Use descriptive names that clearly indicate the purpose of variables, functions, and constants. * **Don't Do This:** Use single-letter variable names (except for loop counters like "i", "j", "k"), cryptic abbreviations, or names that are misleading. * **Why:** Improves readability and comprehension. ### 2.2. Variables * **Do This:** Use "snake_case" for variable names. * **Don't Do This:** Use "camelCase", "PascalCase", or uppercase. * **Why:** Consistent with the existing codebase and common C practice. """c // Correct variable naming int user_id; char *user_name; size_t data_length; // Incorrect variable naming int userID; // camelCase int UserId; // PascalCase int USER_ID; // Uppercase (usually for constants) int x; // Undescriptive """ ### 2.3. Functions * **Do This:** Use "snake_case" for function names. Functions that are internally used (private) within a module or source file should be prefixed with "module_". * **Don't Do This:** Use "camelCase", "PascalCase", or uppercase function names. Do not use global functions unless absolutely necessary. * **Why:** Consistent with the coding style. Module prefix avoids name collisions and clearly indicates function scope. """c // Correct function naming void process_input_data(char *input); int calculate_average(int *data, int count); void module_internal_function(void); //File-scope internal utility function. // Incorrect function naming void ProcessInputData(char *input); // PascalCase void processInputData(char *input); // camelCase void PROCESS_INPUT(char *input); // Uppercase """ ### 2.4. Constants and Macros * **Do This:** Use uppercase with "snake_case" for constants and macros. * **Don't Do This:** Use lowercase or camelCase for constants and macros. * **Why:** Convention to distinguish constants and macros from variables. """c // Correct constant naming #define MAX_CONNECTIONS 1024 const int DEFAULT_PORT = 6379; // Incorrect constant naming #define maxConnections 1024 const int defaultPort = 6379; """ ### 2.5. Data Structures (structs, enums, unions) * **Do This:** Use "PascalCase" for struct, enum, and union names. * **Don't Do This:** Use "snake_case" or lowercase. * **Why:** Consistent with common C practices for type definitions. """c // Correct data structure naming typedef struct UserProfile { int user_id; char *user_name; } UserProfile; typedef enum ConnectionState { CONNECTED, DISCONNECTED, CONNECTING } ConnectionState; // Incorrect data structure naming typedef struct user_profile { int user_id; char *user_name; } user_profile; """ ## 3. Stylistic Consistency Maintaining a consistent style enhances readability and reduces cognitive load. ### 3.1. Comments * **Do This:** * Write clear and concise comments to explain complex logic, algorithms, and data structures. * Use block comments ("/* ... */") for multi-line comments and function headers. * Use single-line comments ("// ...") for brief explanations within a function. * Document the purpose, arguments, and return value of each function. * **Don't Do This:** * Write unnecessary or redundant comments that simply repeat the code. * Use cryptic or ambiguous comments. * Fail to update comments when the code changes. * **Why:** Comments provide context and help others understand the code. """c // Correct comment usage /* * This function calculates the average of an array of integers. * * Parameters: * data: A pointer to the array of integers. * count: The number of elements in the array. * * Returns: * The average of the integers, or 0 if the array is empty. */ int calculate_average(int *data, int count) { if (count == 0) { return 0; // Return 0 for an empty array. } int sum = 0; for (int i = 0; i < count; i++) { sum += data[i]; } return sum / count; } // Incorrect comment usage int calculate_average(int *data, int count) { if (count == 0) { return 0; } int sum = 0; for (int i = 0; i < count; i++) { sum += data[i]; } return sum / count; // Returns the average } """ ### 3.2. Error Handling * **Do This:** * Check for errors and handle them gracefully. Use return values or error codes to indicate success or failure. * Log error messages with sufficient context to aid in debugging. * Clean up resources (memory, file descriptors, etc.) before returning from an error condition. * **Don't Do This:** * Ignore errors or assume that functions always succeed. * Print error messages to stdout without context. * Leak resources on error. * **Why:** Robust error handling is crucial for stability and reliability. """c // Correct Error Handling #include <stdio.h> #include <stdlib.h> int allocate_memory(size_t size, void **ptr) { *ptr = malloc(size); if (*ptr == NULL) { perror("Failed to allocate memory"); // Correct error message return -1; // Correct return value for error } return 0; // Correct return value for success } int main() { void *data; if (allocate_memory(1024, &data) != 0) { fprintf(stderr, "Memory allocation failed.\n"); // Proper error reporting goes to stderr. return 1; // Use non-zero return code to signal failure. } // ... use data ... free(data); return 0; } // Incorrect error handling #include <stdio.h> #include <stdlib.h> int allocate_memory(size_t size, void **ptr) { *ptr = malloc(size); return 0; // Always return success, even if malloc fails } int main() { void *data; allocate_memory(1024, &data); // No error checking // ... use data (potential crash if malloc failed) ... free(data); return 0; } """ ### 3.3. Memory Management * **Do This:** * Allocate and free memory carefully, using "malloc", "calloc", "realloc", and "free". * Always free memory when it is no longer needed to prevent memory leaks. * Use "valgrind" or similar tools to detect memory leaks and errors. * **Don't Do This:** * Forget to free allocated memory. * Free the same memory multiple times (double-free). * Use memory after it has been freed (use-after-free). * **Why:** Proper memory management is essential for preventing crashes and performance degradation. Redis, being a performance-critical application, requires meticulous memory handling. """c // Correct memory management #include <stdlib.h> char *duplicate_string(const char *str) { size_t length = strlen(str) + 1; char *duplicate = (char *)malloc(length); if (duplicate == NULL) { return NULL; // Handle allocation failure } strcpy(duplicate, str); return duplicate; } int main() { char *original = "Redis"; char *copy = duplicate_string(original); if (copy != NULL) { printf("Copy: %s\n", copy); free(copy); // Correctly free the allocated memory } return 0; } // Incorrect memory management #include <stdlib.h> char *duplicate_string(const char *str) { size_t length = strlen(str) + 1; char *duplicate = (char *)malloc(length); strcpy(duplicate, str); return duplicate; } int main() { char *original = "Redis"; char *copy = duplicate_string(original); // Memory allocated for 'copy' is never freed (memory leak) return 0; } """ ### 3.4. Data Structures * **Do This**: Use appropriate data structures for the task at hand. Redis makes extensive uses of "sds" strings, linked lists via "list", dictionaries via "dict", and skiplists. Favor these native data structures where appropriate for optimized performance. * **Don't Do This**: Reimplement existing data structures or use less efficient options. * **Why**: Leverages optimized, well-tested components, contributing to performance and stability. """c // Correct use of SDS strings in Redis #include "sds.h" //Include the sds implementation. int main() { sds my_string = sdsnew("Hello, Redis!"); printf("SDS string: %s\n", my_string); my_string = sdscat(my_string, " - Updated"); // Append to the string safely printf("Updated SDS string: %s\n", my_string); sdsfree(my_string); // Free the SDS string return 0; } // Incorrect (less efficient) use of standard char * #include <stdio.h> #include <stdlib.h> #include <string.h> int main() { char *my_string = malloc(strlen("Hello, Redis!") + 1); strcpy(my_string, "Hello, Redis!"); printf("C string: %s\n", my_string); // Appending requires manual memory management and is prone to errors char *temp = realloc(my_string, strlen(my_string) + strlen(" - Updated") + 1); if (temp == NULL) { free(my_string); // Prevent memory leak on realloc failure. return 1; } my_string = temp; strcat(my_string, " - Updated"); printf("Updated C string: %s\n", my_string); free(my_string); // Free the C string return 0; } """ ## 4. Redis-Specific Considerations These are styles and conventions that are specific to core Redis server development. ### 4.1. Event Loop Integration * **Do This:** Integrate new functionality smoothly with the Redis event loop. Use "aeCreateFileEvent" to register file descriptors for reading and writing events. Avoid blocking operations; prefer asynchronous models. * **Don't Do This:** Introduce blocking calls that can halt the entire server. * **Why:** Redis's performance relies heavily on its non-blocking event loop. """c // Example: Adding a new file event to the Redis event loop (simplified) #include "ae.h" //Include the ae implementation void my_read_handler(aeEventLoop *loop, int fd, void *clientData, int mask) { // Handle data received on the file descriptor printf("Data received on fd %d\n", fd); } int initialize_my_feature(aeEventLoop *loop, int fd) { // Create a file event to listen for readability on the file descriptor if (aeCreateFileEvent(loop, fd, AE_READABLE, my_read_handler, NULL) == AE_ERR) { fprintf(stderr, "Error creating file event\n"); return -1; } return 0; } """ ### 4.2. Command Implementation * **Do This**: Redis commands should be implemented as standalone functions that take a "client" struct as their primary argument. Follow the existing command structure for argument parsing, validation, and reply formatting. Use "addReply*" functions to construct the response. Careful adherence to the existing command structure is extremely important. * **Don't Do This**: Hardcode command logic directly into the event loop or create commands with inconsistent argument handling. * **Why:** Ensures commands are easily manageable, testable, and well-integrated with the Redis protocol. """c // Example: Implementing a simple Redis command (simplified) #include "server.h" void mycommandCommand(client *c) { // Check the number of arguments. Example will require zero arguments if (c->argc != 1) { addReplyErrorArity(c); return; } // Command logic - in this example simply returns "OK" if the call succeeds addReplyString(c, "OK"); //Reply with a simple string result and no error } //Add to the commmand table inside populateCommandTable() using the following pattern: // {"mycommand",mycommandCommand,-1,"readonly",0,NULL,0,0,0,0,0} """ ### 4.3. Configuration Handling * **Do This:** Use the existing configuration framework in "server.c" to manage new configuration options. Define appropriate defaults and validation logic for each option. * **Don't Do This:** Introduce hardcoded configuration values or bypass the configuration system. * **Why:** Centralized configuration management provides consistency and simplifies administration. ### 4.4. Logging * **Do This**: Utilize the "serverLog()" function for all logging within Redis. Choose the appropriate log level (e.g., "LL_DEBUG", "LL_VERBOSE", "LL_WARNING", "LL_NOTICE", "LL_ERROR") based on the severity of the message. * **Don't Do This**: Use "printf()" directly, or choose inappropriate log levels. * **Why**: Enables consistent logging, filtering, and analysis. """c // Example logging with serverLog #include "server.h" // Include the server declarations void my_function(int value) { if (value < 0) { serverLog(LL_WARNING, "Received negative value: %d", value); return; } serverLog(LL_DEBUG, "Processing value: %d", value); // ... rest of the function } """ ### 4.5. Data Type Handling * **Do This:** Use the built-in Redis data type abstractions (e.g., "robj") for storing and manipulating data. These abstractions handle memory management, encoding, and other details. * **Don't Do This:** Directly manipulate raw memory or bypass the data type system. * **Why**. Ensures consistent and efficient data handling throughout the server. """c //Correct use of redis objects. #include "server.h" void my_function(redisDb *db, robj *key, robj *value){ //Value will depend on what is being passed in in key and value, // but it will always be a "robj" data type. dictAdd(db->dict, key, value); //Example function. } """ ## 5. Security Best Practices Security is paramount in Redis development. ### 5.1. Input Validation * **Do This:** Thoroughly validate all input data to prevent buffer overflows, format string vulnerabilities, and other security issues. Use safe string handling functions like "sdscat" and "sdscats". * **Don't Do This:** Trust input data without validation, or use unsafe string functions like "strcpy". * **Why:** Input validation is crucial for preventing security exploits. ### 5.2. Privilege Separation * **Do This:** Run Redis with the lowest possible privileges. Use a dedicated user account for Redis and disable unnecessary features. * **Don't Do This:** Run Redis as root or with excessive privileges. * **Why:** Limits the impact of potential security breaches. ### 5.3. Avoid Deprecated Features * **Do This:** AVOID using deprecated or known vulnerable features unless absolutely necessary. Always check the release notes for security vulnerabilities. * **Don't Do This:** Use deprecated functions that have been replaced with safer alternatives. * **Why:** Using up-to-date code will help prevent security flaws. ### 5.4. Code Review * **Do This:** Have your code reviewed by other experienced Redis developers before committing changes. Code review is a crucial step in identifying potential security vulnerabilities and other issues. * **Don't Do This:** Push code without review. * **Why** Increased security and code confidence. ## 6. Testing * **Do This**: Write comprehensive unit tests and integration tests to verify the correctness and performance of new code. Use the existing Redis testing framework. * **Don't Do This**: Neglect testing, or rely solely on manual testing. * **Why**: Automated testing ensures code quality and prevents regressions.

# Core Architecture Standards for Redis This document outlines the core architectural standards for Redis development. It aims to guide developers in building maintainable, performant, and secure Redis applications. These standards reflect modern best practices based on the latest version of Redis (at the time of writing, this is considered to be Redis 7.x, though upcoming versions will have their considerations). The focus is to establish a consistent approach to project structure, architectural patterns, and implementation details, enabling a cohesive and high-quality codebase. ## 1. Project Structure and Organization A well-defined project structure promotes code discoverability, testability, and maintainability. ### 1.1. Standard Directory Layout **Do This:** Adhere to a consistent directory structure, separating concerns. """ redis-module/ ├── src/ # Source code files: C code, API implementations ├── include/ # Header files: Module definitions, API declarations ├── tests/ # Test suite: Unit tests, integration tests ├── deps/ # Third-party dependencies (if any, highly discouraged to modify) ├── Makefile # Build instructions ├── README.md # Project description and setup instructions ├── LICENSE # Licensing information └── config.mk # Configuration parameters for compilation """ **Don't Do This:** Intermix source code, tests, and build scripts in a single directory. **Why:** Clean separation of concerns simplifies navigation, compilation, and testing. Modifications to tests or build configurations are less likely to unintentionally affect the core source code. **Code Example:** """makefile # Example Makefile excerpt CFLAGS = -Wall -O2 -g -I./include -I./deps/hiredis LDFLAGS = -shared redismodule.so: src/*.c $(CC) $(CFLAGS) -fPIC $^ -o $@ $(LDFLAGS) test: redismodule.so tests/*.c $(CC) $(CFLAGS) -I./include -L. -lredismodule $^ -o test_runner ./test_runner """ ### 1.2. Module Decomposition **Do This:** Decompose large modules into smaller, functionally cohesive units. **Don't Do This:** Create monolithic modules containing unrelated functionality. **Why:** Smaller modules are easier to understand, test, and reuse. This improves code maintainability and reduces the likelihood of introducing bugs during modifications. **Code Example:** Imagine a module with extensive string manipulation functions. Break it down: """c // string_utils.h #ifndef STRING_UTILS_H #define STRING_UTILS_H #include <stddef.h> char *string_duplicate(const char *str); size_t string_length(const char *str); // ... other string utility function declarations #endif """ """c // string_utils.c #include "string_utils.h" #include <stdlib.h> #include <string.h> char *string_duplicate(const char *str) { size_t len = string_length(str); char *new_str = malloc(len + 1); if (new_str == NULL) return NULL; // Handle allocation failure memcpy(new_str, str, len + 1); return new_str; } size_t string_length(const char *str) { return strlen(str); } // ... other string utility function implementations """ This separation allows "string_utils" to be tested independently and reused in other parts of your module or even different modules. ## 2. Architectural Patterns Choosing appropriate architectural patterns is crucial for scalability, maintainability, and performance. ### 2.1. Event-Driven Architecture **Do This:** Leverage Redis's Pub/Sub functionality for asynchronous communication between services. **Don't Do This:** Rely solely on synchronous request-response patterns when asynchronous alternatives exist. **Why:** Event-driven architectures decouple services, improve responsiveness, and enable scalability. Pub/Sub allows services to react to events without direct dependencies on the event source. **Code Example:** Using Redis Pub/Sub with a simple publisher and subscriber. """python # Publisher (Python example using redis-py) import redis import time r = redis.Redis(host='localhost', port=6379, db=0) for i in range(10): r.publish('my_channel', f'Message {i}') print(f"Published: Message {i}") time.sleep(1) """ """python # Subscriber (Python example using redis-py) import redis r = redis.Redis(host='localhost', port=6379, db=0) pubsub = r.pubsub() pubsub.subscribe('my_channel') for message in pubsub.listen(): if message['type'] == 'message': print(f"Received: {message['data'].decode('utf-8')}") """ ### 2.2. Data Locality **Do This:** Design your data model to maximize data locality within Redis. Use appropriate data structures to keep related data together. **Don't Do This:** Spread related data across multiple keys unnecessarily, leading to increased network latency and lookup overhead. **Why:** Data locality improves performance by reducing the number of round trips to the Redis server. Using appropriate data structures (e.g., Hashes, Lists, Sets, Sorted Sets, Streams) allows you to group related data within a single key. **Code Example:** Storing user profile data in a Hash instead of individual keys. """redis # Anti-pattern: Storing user data in separate keys SET user:123:name "John Doe" SET user:123:email "john.doe@example.com" SET user:123:age 30 # Best practice: Storing user data in a Hash HSET user:123 name "John Doe" email "john.doe@example.com" age 30 # Retrieving all user data with a single command HGETALL user:123 """ Maintaining data locality minimizes the network round trips. For more complex relationships consider RedisJSON and RedisGraph modules. ### 2.3. Caching Strategies **Do This:** Implement caching strategies appropriate for your application's needs (e.g., write-through, write-back, cache-aside). Carefully consider TTLs and cache invalidation policies. **Don't Do This:** Use Redis solely as a persistent data store without leveraging its caching capabilities. Neglect to implement proper cache invalidation, leading to stale data. **Why:** Redis is primarily an in-memory data store optimized for caching. Leveraging caching strategies improves performance and reduces the load on persistent databases. **Code Example:** Implementing a cache-aside pattern in Python. """python import redis import json import time r = redis.Redis(host='localhost', port=6379, db=0) def get_user_profile(user_id, get_from_db): # get_from_db is a function to fetch from DB cache_key = f"user:{user_id}:profile" cached_data = r.get(cache_key) if cached_data: print("Fetching from cache") return json.loads(cached_data.decode('utf-8')) # Decode bytes print("Fetching from database") user_profile = get_from_db(user_id) # Assuming a function to fetch from DB r.setex(cache_key, 3600, json.dumps(user_profile)) #setex for TTL return user_profile # Example Usage(replace get_user_from_db with your function): # assuming you have a function to fetch the profile from the db # def get_user_profile_from_db(user_id): # # DB access code here using SQL or similar to fetch the profile # return user_profile_data #profile = get_user_profile(123, get_user_profile_from_db) #print(profile) """ ### 2.4 Redis Streams for Eventing **Do This:** Use Redis Streams for reliable, persistent event logging and complex event processing. Groups and consumer management within Streams avoid message loss. **Don't Do This:** Rely solely on Pub/Sub for asynchronous eventing where message delivery guarantees are required. Overcomplicate fan-out patterns with manual management when groups can handle it. **Why:** Streams provides at-least-once delivery, persistence, consumer groups, and the ability for clients to consume messages in a reliable fashion. This functionality is ideal for event sourcing, change data capture (CDC), and queueing applications. **Code Example**: Reading a message from a consumer group: """python import redis redis_client = redis.Redis(host='localhost', port=6379) stream_key = 'my_stream' group_name = 'my_group' consumer_name = 'consumer_1' # Create stream group try: redis_client.xgroup_create(stream_key, group_name, id='0') except redis.exceptions.ResponseError as e: if str(e) == 'BUSYGROUP Consumer Group name already exists': print("Consumer group already exists") else: raise e """ """python # Consume messages from the group while True: response = redis_client.xreadgroup(groupname=group_name, consumername=consumer_name, streams={stream_key: '>'}, count=1, block=5000) #Block indefinitely for a new message if response: stream_name, messages = response[0] message_id, message_data = messages[0] print(f"Received message: {message_data}") redis_client.xack(stream_key, group_name, message_id) # Acknowledge the message else: print("No new messages.") """ ### 2.5 Data Modeling in Redis. **Do This**: Choose the appropriate data structure considering access patterns, data size, and operations. **Don't Do This**: Using only STRINGS for everything. **Why**: Redis offers multiple data structures engineered for speed based on the specific operations they optimize. **Code Example**: * **Strings**: Simple key-value storage. * **Lists**: Ordered collections, suited for queues or time series. * **Sets**: Unique, unordered collections, great for membership testing. * **Hashes**: Record-like structures within a key. * **Sorted Sets**: Ordered sets with scoring, perfect for leaderboards and range queries. * **Streams:** Append-only log; messaging and event sourcing. * **Geospatial Indexes**: Store and query geographic data. * **Bitmaps & HyperLogLogs**: Space-efficient data structures for specific tasks. Example using a ZSET for a leaderboard: """redis ZADD leaderboard 100 "player1" ZADD leaderboard 150 "player2" ZADD leaderboard 120 "player3" ZRANGE leaderboard 0 -1 REV # Fetch the leaderboard in descending order """ ## 3. Implementation Details Adhering to consistent coding conventions and best practices improves code readability, maintainability, and collaboration. ### 3.1. Error Handling **Do This:** Implement robust error handling. Check return values of Redis commands and handle potential errors gracefully. **Don't Do This:** Ignore error conditions, leading to unpredictable behavior and potential data corruption. **Why:** Proper error handling prevents application crashes and ensures data integrity. Log errors appropriately for debugging and monitoring. **Code Example:** Example in C (using hiredis). This is fundamental for module development. """c #include <stdio.h> #include <stdlib.h> #include <string.h> #include <hiredis/hiredis.h> int main() { redisContext *c = redisConnect("localhost", 6379); if (c == NULL || c->err) { if (c) { printf("Connection error: %s\n", c->errstr); redisFree(c); } else { printf("Connection error: can't allocate redis context\n"); } exit(1); } redisReply *reply = redisCommand(c, "SET mykey myvalue"); if (reply == NULL) { printf("Error executing command: %s\n", c->errstr); redisFree(c); exit(1); } if (reply->type == REDIS_REPLY_ERROR) { printf("Redis error: %s\n", reply->str); } else { printf("SET command executed successfully\n"); } freeReplyObject(reply); redisFree(c); return 0; } """ ### 3.2. Memory Management **Do This:** Carefully manage memory allocation and deallocation. Avoid memory leaks and use appropriate data structures to minimize memory consumption. **Don't Do This:** Leak memory, leading to performance degradation and application instability. Use excessive memory without considering alternatives like Redis's optimized data structures. **Why:** Redis is an in-memory data store, and memory management is crucial for its performance and stability. Leaked memory can eventually lead to application crashes. Efficient memory usage maximizes the amount of data that can be stored in Redis. **Code Example:** String duplication with "strdup" and explicit "free". """c #include <stdlib.h> #include <string.h> char *duplicate_string(const char *str) { char *new_str = strdup(str); // Duplicate string, allocating memory if (new_str == NULL) { // Handle allocation failure (e.g., log error, return NULL) return NULL; } return new_str; } void free_string(char *str) { if (str != NULL) { free(str); // Deallocate memory } } //Usage: /* char *my_string = duplicate_string("Hello, Redis!"); if (my_string == NULL){ //Handle case of memory allocation failure } // ... use my_string ... free_string(my_string); //When done! */ """ ### 3.3. Logging **Do This:** Implement structured and informative logging to track application behavior and diagnose issues. **Don't Do This:** Use excessive or insufficient logging. Log sensitive information inappropriately. **Why:** Logs are essential for monitoring, debugging, and auditing. Structured logging (e.g., using JSON format) simplifies log analysis. **Code Example:** Basic logging with varying severity levels. """python import logging logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s') logging.debug('This is a debug message') # For detailed development information logging.info('This is an informational message') # Normal operations logging.warning('This is a warning message') # Potential issues logging.error('This is an error message') # Something went wrong logging.critical('This is a critical message') # Serious issue needing immediate attention try: result = 10 / 0 except Exception as e: logging.exception("An exception occurred") # Logs full stack trace """ ### 3.4. Concurrency and Atomicity **Do This:** Use Redis's atomic commands and transactions to ensure data consistency in concurrent environments. Employ optimistic locking with "WATCH" for complex operations. **Don't Do This:** Neglect concurrency issues, leading to race conditions and data corruption. Manually implement locking mechanisms when Redis's built-in features can provide atomic operations. **Why:** Redis is single-threaded, but concurrent clients can still cause race conditions if operations are not atomic. Transactions and "WATCH" provide mechanisms for ensuring data consistency in these scenarios. **Code Example:** Optimistic locking using "WATCH". This allows you to control changes during a multi-stage update. """python import redis r = redis.Redis(host='localhost', port=6379, db=0) def increment_counter(counter_key): while True: try: # Watch key for changes made by other clients between GET and SET with r.lock('increment_lock', blocking_timeout=5): #Example using lock (distributed) r.watch(counter_key) # Start watching the key current_value = r.get(counter_key) # Get current value if current_value is not None: new_value = int(current_value) + 1 else: new_value = 1 # Set to 1 if the key doesn't exist # Start a transaction and attempt to increment atomically pipe = r.pipeline() pipe.multi() # Start the transaction pipe.set(counter_key, new_value) # Set the new value result = pipe.execute() # Executes all commands in the pipeline atomically, returning the results if result: # result will be [True] if the transaction succeeded print(f"Counter incremented to {new_value}") return True except redis.WatchError: print("The key changed! Retrying...") # Key changed between GET and SET; retry the operation except redis.exceptions.LockError: print("Couldn't acquire lock - is another process updating the value?") return False #Or retry. finally: if r.connection: r.unwatch() # Stop watching (happens automatically with pipe.execute) """ ### 3.5. Use of Redis Modules **Do This:** When appropriate, leverage Redis Modules (e.g., RedisJSON, RedisSearch, RedisGraph) to extend Redis functionality and optimize performance. **Don't Do This:** Re-implement functionality readily available in existing Redis Modules. Use modules without properly understanding their impact on performance and stability. **Why:** Modules provide powerful extensions to Redis, enabling specialized data structures and query capabilities. They can significantly improve performance for specific use cases compared to implementing the same functionality in application code. **Example: Using RedisJSON** """python import redis import redis.commands.json.path as jsonpath r = redis.Redis(host='localhost', port=6379) r.json().set('user:123', jsonpath.ROOT_PATH, { 'name': 'John Doe', 'email': 'john.doe@example.com', 'age': 30 }) user_data = r.json().get('user:123', jsonpath.ROOT_PATH) print(user_data) # Output: {'name': 'John Doe', 'email': 'john.doe@example.com', 'age': 30} age = r.json().get('user:123', '.age') print(age) # Output 30 """ ### 3.6. Connection Management **Do This:** Use connection pooling and reuse connections to minimize connection overhead, especially with the latest client libraries. **Don't Do This:** Create a new Redis connection for every operation, which can be very expensive. **Why:** Establishing a new Redis connection has overhead. Connection reuse can significantly reduce the latency of Redis operations, particularly in frequently accessed parts of the application. Most modern Redis clients implement connection pooling. **Example:** Using connection pooling in Python (redis-py). """python import redis # Create a connection pool pool = redis.ConnectionPool(host='localhost', port=6379, db=0, max_connections=10) # Get a connection from the pool r = redis.Redis(connection_pool=pool) # Perform operations r.set('mykey', 'myvalue') value = r.get('mykey') print(value) #The connection will be returned to the pool when it goes out of scope. """ ## 4. Monitoring and Observability Effective monitoring and alerting are crucial for maintaining a healthy Redis deployment. ### 4.1. Key Performance Indicators (KPIs) **Do This:** Monitor key Redis metrics, such as memory usage, CPU utilization, connection count, and command latency. **Don't Do This:** Operate Redis without monitoring, making it difficult to identify and resolve performance bottlenecks or issues. **Why:** Monitoring KPIs allows you to proactively identify potential problems, optimize Redis configuration, and ensure the application's performance meets expectations. ### 4.2. Alerting **Do This:** Configure alerts based on critical metrics to notify you of potential issues, such as high memory usage or slow command latency. **Don't Do This:** Ignore critical alerts, leading to prolonged outages or performance degradation. **Why:** Alerting allows you to quickly respond to issues and prevent them from escalating into major problems. ### 4.3. Logging and Tracing **Do This:** Integrate distributed tracing to track requests across multiple services, including Redis. Use structured logging to easily query and analyze log data. **Don't Do This:** Rely solely on Redis logs for debugging, which can be difficult to correlate with application-level events. **Why:** Distributed tracing improves end-to-end visibility into application performance. Structured logging enables efficient log analysis and correlation. By adhering to these core architectural standards, Redis developers can build robust, performant, and maintainable applications that fully leverage the power of Redis. Continuous review and improvement of these standards are essential to adapting to new Redis features and evolving best practices.

# API Integration Standards for Redis This document outlines the coding standards for integrating Redis with backend services and external APIs. It provides guidelines for developers to ensure maintainable, performant, and secure integration practices. ## 1. General Principles ### 1.1. Use Case Driven Design **Do This:** Design API integrations around specific use cases. Redis should serve a well-defined purpose within the broader architecture, whether caching API responses, acting as a message broker, or managing rate limits. **Don't Do This:** Treat Redis as a generic data store without a clear understanding of how it fits into the API landscape. Avoid pushing data into Redis without considering the query patterns. **Why:** A use-case driven approach ensures that the Redis data structures and operations are optimized for the intended purpose, improving performance and reducing unnecessary resource consumption. **Example:** * **Good:** Caching weather API responses based on geographical coordinates to reduce latency. * **Bad:** Storing miscellaneous API data in Redis without a clear access pattern or expiration strategy. ### 1.2. Loose Coupling **Do This:** Decouple the Redis layer from the API logic as much as possible. Use abstract interfaces or services to interact with Redis. **Don't Do This:** Directly embed Redis operations within API handlers or backend service code, creating tight dependencies. **Why:** Decoupling makes the API more flexible and easier to maintain. You can change the underlying Redis implementation (e.g., switch providers, scale the cluster) without affecting the API logic. **Example:** """python # Good: Using an abstract service layer class CacheService: def get_data(self, key): raise NotImplementedError def set_data(self, key, value, expiry=None): raise NotImplementedError class RedisCacheService(CacheService): def __init__(self, redis_client): self.redis_client = redis_client def get_data(self, key): return self.redis_client.get(key) def set_data(self, key, value, expiry=None): self.redis_client.set(key, value, ex=expiry) # API Handler def get_resource(resource_id, cache_service: CacheService): cached_data = cache_service.get_data(f"resource:{resource_id}") if cached_data: return cached_data # Fetch from backend, cache, and return data_from_backend = fetch_from_backend(resource_id) cache_service.set_data(f"resource:{resource_id}", data_from_backend, expiry=3600) # 1 hour return data_from_backend # Bad: Tightly coupled Redis operations inside API handler import redis redis_client = redis.Redis(host='localhost', port=6379) def get_resource_tightly_coupled(resource_id): cached_data = redis_client.get(f"resource:{resource_id}") if cached_data: return cached_data # Fetch from backend, cache, and return data_from_backend = fetch_from_backend(resource_id) redis_client.set(f"resource:{resource_id}", data_from_backend, ex=3600) # 1 hour return data_from_backend """ ### 1.3. Error Handling **Do This:** Implement robust error handling for Redis interactions. Catch exceptions, log errors, and gracefully degrade functionality if Redis is unavailable. Use retry mechanisms with exponential backoff for transient errors. **Don't Do This:** Silently ignore Redis errors or propagate exceptions to the API client without proper handling. **Why:** Redis failures should not crash the API. Proper error handling ensures resilience and provides valuable insights into potential issues. **Example:** """python import redis import time import logging logging.basicConfig(level=logging.INFO) def get_data_with_retry(key, redis_client, max_retries=3, initial_delay=0.1): for attempt in range(max_retries): try: data = redis_client.get(key) return data except redis.exceptions.ConnectionError as e: logging.error(f"Redis connection error (attempt {attempt + 1}/{max_retries}): {e}") if attempt == max_retries - 1: logging.error("Max retries reached. Unable to connect to Redis.") return None delay = initial_delay * (2 ** attempt) # Exponential backoff logging.info(f"Retrying in {delay:.2f} seconds...") time.sleep(delay) except redis.exceptions.RedisError as e: logging.error(f"Redis error: {e}") return None # Non-retriable error """ ## 2. API Caching ### 2.1. Cache-Aside Pattern **Do This:** Use the Cache-Aside pattern for reading data. Check Redis first, and if the data is not present (cache miss), retrieve it from the backend, store it in Redis, and then return it. **Don't Do This:** Directly read from the backend every time, bypassing Redis cache. **Why:** The Cache-Aside pattern minimizes the load on the backend database and improves API response times. **Example:** """python import redis import json redis_client = redis.Redis(host='localhost', port=6379) def get_user_profile(user_id): cache_key = f"user:{user_id}:profile" cached_profile = redis_client.get(cache_key) if cached_profile: logging.info(f"Cache hit for user {user_id}") return json.loads(cached_profile.decode('utf-8')) #decode bytes to string, then convert to dict logging.info(f"Cache miss for user {user_id}. Fetching from backend.") profile_data = fetch_user_profile_from_db(user_id) # Simulate fetching from database if profile_data: redis_client.set(cache_key, json.dumps(profile_data), ex=3600) # Cache for 1 hour return profile_data else: return None # return None if the user does not exist def fetch_user_profile_from_db(user_id): # Simulate fetching data from the database if user_id == "123": return {"user_id": "123", "name": "John Doe", "email": "john.doe@example.com"} else: return None #Simulate API call profile = get_user_profile("123") print(profile) #Cache Hit demonstration profile = get_user_profile("123") print(profile) """ ### 2.2. Cache Invalidation Strategies **Do This:** Implement an appropriate cache invalidation strategy based on the data's volatility. Use TTLs (Time-To-Live) for time-based invalidation. For event-driven invalidation, consider using Redis Pub/Sub or Redis Streams to notify other services when data changes in the backend. **Don't Do This:** Rely solely on fixed TTLs without considering data updates in the backend. Allow stale data to persist indefinitely. Manually delete keys without considering atomicity. **Why:** Proper cache invalidation ensures data consistency between the cache and the backend. Stale data can lead to incorrect API responses. **Example (TTL and Event-Driven Invalidation):** """python import redis import json redis_client = redis.Redis(host='localhost', port=6379) def update_user_profile(user_id, new_data): # Update data in backend database first (this is a simulation) update_user_profile_in_db(user_id, new_data) # Invalidate the cache cache_key = f"user:{user_id}:profile" redis_client.delete(cache_key) # Delete the cached key # Publish a message to Redis Pub/Sub to notify other services redis_client.publish("user_profile_updates", json.dumps({"user_id": user_id})) def update_user_profile_in_db(user_id, new_data): # Simulate updating user profile in the database logging.info(f"Simulating database update for user {user_id} with new data: {new_data}") # Example of subscriber for the Pub/Sub channel def subscribe_to_profile_updates(): pubsub = redis_client.pubsub() pubsub.subscribe("user_profile_updates") for message in pubsub.listen(): if message["type"] == "message": data = json.loads(message["data"].decode("utf-8")) user_id = data["user_id"] # Invalidate local in-memory cache or perform other actions logging.info(f"Received update for user {user_id}. Invalidating local cache.") import threading # Start subscriber in a separate thread (This should go in its own service) subscriber_thread = threading.Thread(target=subscribe_to_profile_updates) subscriber_thread.daemon = True # Allow main program to exit if only daemon threads are running subscriber_thread.start() update_user_profile("123", {"name": "Updated Name", "email": "updated@example.com"}) #Simulate API call to update the profile """ ### 2.3. Conditional Updates using WATCH **Do This:** Utilize the "WATCH" command for optimistic locking when updating cached data based on its previous value. This ensures atomicity and prevents race conditions. **Don't Do This:** Perform read-modify-write operations on cached data without any concurrency control. **Why:** Without optimistic locking, concurrent updates can lead to data corruption, especially when incrementing counters or modifying complex data structures. **Example:** """python import redis import time redis_client = redis.Redis(host='localhost', port=6379) def increment_counter(counter_key): while True: try: # WATCH the key to monitor for modifications redis_client.watch(counter_key) # Get current value (within the transaction) current_value = redis_client.get(counter_key) if current_value is None: current_value = 0 # Initialize if it doesn't exist else: current_value = int(current_value) # Start a new transaction pipe = redis_client.pipeline() # Increment the value new_value = current_value + 1 pipe.set(counter_key, new_value) # Execute the transaction pipe.execute() logging.info(f"Counter '{counter_key}' incremented successfully to {new_value}.") break # Exit the loop if the transaction was successful except redis.exceptions.WatchError: # Another client modified the key; retry the operation logging.warning(f"WatchError: Counter '{counter_key}' was modified by another client. Retrying...") time.sleep(0.1) # Add a small delay before retrying finally: # Unwatch the key, regardless of success or failure in transaction redis_client.unwatch() """ ## 3. Rate Limiting ### 3.1. Token Bucket Algorithm **Do This:** Implement rate limiting using the token bucket algorithm with Redis. Use Lua scripts for atomic operations to ensure accuracy and prevent race conditions. Consider RedisTimeSeries for more advanced rate limiting and analytics. **Don't Do This:** Implement rate limiting logic in the application code without using atomic operations. **Why:** The token bucket algorithm provides a flexible and efficient way to control the rate at which requests are processed. Lua scripts ensure atomicity, preventing race conditions when multiple clients access the same rate limiter. **Example:** """python import redis import time redis_client = redis.Redis(host='localhost', port=6379) # Lua script for token bucket implementation TOKEN_BUCKET_SCRIPT = """ local key = KEYS[1] local capacity = tonumber(ARGV[1]) local refill_rate = tonumber(ARGV[2]) local now = tonumber(ARGV[3]) local cost = tonumber(ARGV[4]) local last_refill_timestamp = redis.call('hget', key, 'last_refill_timestamp') local tokens = redis.call('hget', key, 'tokens') if last_refill_timestamp == false then last_refill_timestamp = now tokens = capacity else last_refill_timestamp = tonumber(last_refill_timestamp) tokens = tonumber(tokens) local time_passed = now - last_refill_timestamp local refill = time_passed * refill_rate tokens = math.min(capacity, tokens + refill) end if tokens >= cost then tokens = tokens - cost redis.call('hset', key, 'last_refill_timestamp', now) redis.call('hset', key, 'tokens', tokens) return 1 -- Allow request else redis.call('hset', key, 'last_refill_timestamp', now) -- still update last refill time to avoid high load from retries redis.call('hset', key, 'tokens', tokens) -- update existing token count return 0 -- Deny request end """ # Load the Lua script into Redis token_bucket = redis_client.register_script(TOKEN_BUCKET_SCRIPT) def is_rate_limited(user_id, capacity, refill_rate, cost=1): key = f"rate_limit:{user_id}" now = time.time() allowed = token_bucket(keys=[key], args=[capacity, refill_rate, now, cost]) # Execute Lua script return allowed == 0 # 0 means request is denied # Example Usage user_id = "user123" capacity = 10 # Maximum number of requests refill_rate = 2 # Tokens refilled per second. if is_rate_limited(user_id, capacity, refill_rate): print(f"Request rate limited for user {user_id}") else: print(f"Request allowed for user {user_id}") # Process the request """ ### 3.2. RedisTimeSeries for Advanced Rate Limiting **Do This:** Explore using RedisTimeSeries for more sophisticated rate limiting scenarios, such as tracking request rates over time or implementing sliding window rate limiters. **Don't Do This:** Stick to basic counter-based rate limiting if you need to analyze rate limits over time or implement dynamic adjustments. **Why:** RedisTimeSeries provides built-in commands for time series data management, making it easier to implement complex rate limiting strategies and gather insights into API usage patterns. **Example (Conceptual - Requires RedisTimeSeries module):** """python # Conceptual - Requires the RedisTimeSeries Module # Note: You'd typically use a RedisTimeSeries client library # but this is illustrative of the commands. # Add a time series for the user if it doesn't exist # redis_client.ts().create(f"user:{user_id}:requests") # Requires RedisTimeSeries module # Record the event # redis_client.ts().add(f"user:{user_id}:requests", "*", 1) # Adds a single event # Check rate within the last minute (sliding window) # requests_last_minute = redis_client.ts().range(f"user:{user_id}:requests", "-60s", "+inf") # if len(requests_last_minute) > RATE_LIMIT: # print("Exceeded rate limit") # else: # print("Request allowed") """ ## 4. Message Queuing ### 4.1. Redis Streams **Do This:** Use Redis Streams for reliable message queuing and asynchronous processing of API requests. Leverage consumer groups for parallel processing and fault tolerance. Implement retry mechanisms for failed message processing. **Don't Do This:** Rely solely on simple lists for message queuing, as they lack advanced features like consumer groups, persistence, and acknowledgement. **Why:** Redis Streams provides a durable and ordered message queue with built-in support for consumer groups, making it ideal for asynchronous tasks and decoupling API requests from backend processing. Using streams facilitates building microservices and enhances resilience. **Example:** """python import redis import json redis_client = redis.Redis(host='localhost', port=6379) STREAM_NAME = "api_requests" CONSUMER_GROUP_NAME = "my_group" CONSUMER_NAME = "consumer_1" def enqueue_api_request(request_data): message_id = redis_client.xadd(STREAM_NAME, request_data) logging.info(f"Enqueued message with ID: {message_id.decode('utf-8')}") return message_id def process_message(message_id, message_data): # Simulate API request processing logging.info(f"Processing message ID: {message_id.decode('utf-8')} with data: {message_data}") # Simulate a failure, but do not remove the messages if 'simulated_failure' in message_data and message_data['simulated_failure']: raise Exception("Simulated failure") def consume_messages(): # Create consumer group, if it doesn't exist try: redis_client.xgroup_create(STREAM_NAME, CONSUMER_GROUP_NAME, id='0', mkstream=True) except redis.exceptions.ResponseError as e: if str(e) == 'BUSYGROUP Consumer Group name already exists': logging.info("Consumer group already exists, continuing...") else: raise while True: try: # Read messages from the stream, blocking if there are none messages = redis_client.xreadgroup(groupname=CONSUMER_GROUP_NAME, consumername=CONSUMER_NAME, streams={STREAM_NAME: '>'}, count=1, block=5000) # Block for 5 seconds if messages: stream_name, message_list = messages[0] message_id, message_data = message_list[0] try: process_message(message_id, message_data) # Acknowledge the message after successful processing redis_client.xack(STREAM_NAME, CONSUMER_GROUP_NAME, message_id) logging.info(f"Acknowledged message: {message_id.decode('utf-8')}") except Exception as e: logging.error(f"Error processing message {message_id}: {e}") # Optionally, requeue for another attempt. # A more robust implementation would have retry counts / dead letter queue. # redis_client.xadd(STREAM_NAME, message_data) continue else: logging.info("No new messages for 5 seconds. Checking again...") except redis.exceptions.ConnectionError as e: logging.error(f"Redis connection error: {e}. Retrying in 5 seconds...") time.sleep(5) except Exception as e: logging.error(f"Unexpected error: {e}. Continuing...") import threading # Start consumer in a separate thread consumer_thread = threading.Thread(target=consume_messages) consumer_thread.daemon = True # Allow main program to exit if only daemon threads are running consumer_thread.start() # Simulate API Request enqueue_api_request({"api_endpoint": "/users", "method": "POST", "payload": {"name": "New User"}}) enqueue_api_request({"api_endpoint": "/events", "method": "POST", "payload": {"type": "User Created", "description": "description"}, 'simulated_failure': True}) # Enqueue a request time.sleep(20) #let the requests process """ ### 4.2. Dead Letter Queues (DLQ) **Do This:** Implement a Dead Letter Queue (DLQ) for messages that fail to process after multiple retries in Redis Streams. This allows you to inspect and potentially re-process problematic messages and prevents service outages. **Don't Do This:** Discard failed messages or let them remain in the main stream indefinitely, as this can lead to data loss or service degradation. **Why:** A DLQ provides a mechanism for handling errors gracefully and ensuring that no data is lost. It's especially crucial in distributed systems where message processing failures are common. **Example (DLQ Implementation):** """python import redis import json import time redis_client = redis.Redis(host='localhost', port=6379) STREAM_NAME = "api_requests" DLQ_STREAM_NAME = "api_requests_dlq" CONSUMER_GROUP_NAME = "my_group" CONSUMER_NAME = "consumer_1" MAX_RETRIES = 3 # Helper function to move messages to the DLQ. def move_to_dlq(message_id, message_data, error_message): #Add the error message to DLQ entry dlq_data = message_data dlq_data['error'] = error_message redis_client.xadd(DLQ_STREAM_NAME, dlq_data ) redis_client.xack(STREAM_NAME, CONSUMER_GROUP_NAME, message_id) redis_client.xdel(STREAM_NAME, message_id) logging.info(f"Message {message_id.decode('utf-8')} moved to DLQ.") def process_message(message_id, message_data, retry_count): # Simulate API request processing logging.info(f"Processing message ID: {message_id.decode('utf-8')} with data: {message_data}, attempt #{retry_count}") # Simulate a failure if 'simulated_failure' in message_data and message_data['simulated_failure']: raise Exception("Simulated failure") def consume_messages(): # Create stream and consumer group if they don't exist try: redis_client.xgroup_create(STREAM_NAME, CONSUMER_GROUP_NAME, id='0', mkstream=True) redis_client.xgroup_create(DLQ_STREAM_NAME, CONSUMER_GROUP_NAME, id='0', mkstream=True) # Create DLQ Stream as well except redis.exceptions.ResponseError as e: if str(e) == 'BUSYGROUP Consumer Group name already exists': logging.info("Consumer group already exists, continuing...") else: raise while True: try: #Check for pending messages first pending_messages = redis_client.xpending(STREAM_NAME, CONSUMER_GROUP_NAME, min='-', max='+', count=10) #Check for pending messages. for message_id, consumer_name, time_since_delivery, delivery_count in pending_messages: if delivery_count > MAX_RETRIES: logging.warning(f"Message {message_id.decode('utf-8')} has exceeded max retries ({MAX_RETRIES}). Moving to DLQ.") # fetch the problematic message message_data = redis_client.xrange(STREAM_NAME, message_id, message_id, count=1, reverse=False) _, message_content = message_data[0] #move this message to DLQ move_to_dlq(message_id, message_content, f"Max retries exceeded: {MAX_RETRIES}") # Read messages from the stream, blocking if there are none messages = redis_client.xreadgroup(groupname=CONSUMER_GROUP_NAME, consumername=CONSUMER_NAME, streams={STREAM_NAME: '>'}, count=1, block=5000) # Block for 5 seconds if messages: stream_name, message_list = messages[0] message_id, message_data = message_list[0] retry_count = 1 # reset the retries on each message try: process_message(message_id, message_data, retry_count) # Acknowledge the message after successful processing redis_client.xack(STREAM_NAME, CONSUMER_GROUP_NAME, message_id) logging.info(f"Acknowledged message: {message_id.decode('utf-8')}") except Exception as e: logging.error(f"Error processing message {message_id}: {e}") message_list_from_xinfo = redis_client.xinfo_consumers(STREAM_NAME, groupname=CONSUMER_GROUP_NAME, count=1) # get the message retry_count = 0 for consumer_info in message_list_from_xinfo: retry_count = int(consumer_info['pending']) if consumer_info['pending'] != None else 0 if retry_count > MAX_RETRIES: move_to_dlq(message_id, message_data, "An Error has occurred") else: time.sleep(1) #Simulate a short wait logging.warning(f"Retrying processing message {message_id.decode('utf-8')} (attempt {retry_count + 1})") else: logging.info("No new messages for 5 seconds. Checking again...") except redis.exceptions.ConnectionError as e: logging.error(f"Redis connection error: {e}. Retrying in 5 seconds...") time.sleep(5) except Exception as e: logging.error(f"Unexpected error: {e}. Continuing...") # Start consumer in a separate thread consumer_thread = threading.Thread(target=consume_messages) consumer_thread.daemon = True # Allow main program to exit if only daemon threads are running consumer_thread.start() # Simulate API Request enqueue_api_request({"api_endpoint": "/users", "method": "POST", "payload": {"name": "New User"}}) # Enqueue a request to be moved to dead letter queue enqueue_api_request({"api_endpoint": "/events", "method": "POST", "payload": {"type": "User Created", "description": "description"}, 'simulated_failure': True}) time.sleep(20) """ ## 5. Serialization ### 5.1. Standard Formats **Do This:** Use standard serialization formats like JSON or Protocol Buffers for storing complex data in Redis. Base64 encode binary data. **Don't Do This:** Rely on custom serialization methods or store unstructured text data, making it difficult to parse and maintain. **Why:** Standard serialization formats improve data interoperability, reduce code complexity, and allow you to easily evolve data structures over time. ### 5.2. Compression **Do This:** Compress large cached objects before storing them in Redis to reduce memory usage and network bandwidth. Use libraries like "zlib" or "gzip". **Don't Do This:** Store large uncompressed objects in Redis, especially if you have limited memory resources. **Why:** Compression can significantly reduce the amount of memory required to store cached data, improving Redis performance and scalability. ## 6. Security ### 6.1. Authentication **Do This:** Always enable authentication in Redis by setting a strong password. **Don't Do This:** Run Redis without authentication, exposing it to unauthorized access. **Why:** Authentication protects your Redis instance from malicious actors who could steal or corrupt your data. ### 6.2. Network Isolation **Do This:** Restrict network access to Redis to only the necessary IP addresses or networks. Use firewalls or network security groups to control traffic. **Don't Do This:** Expose Redis to the public internet without proper network isolation. **Why:** Network isolation limits the attack surface and prevents unauthorized access from external sources. **Example (.conf file)** """ bind 127.0.0.1 10.0.0.10 # Only listen on these IP addresses protected-mode yes # This is a good default requirepass your_strong_password # Make sure to have strong password """ ### 6.3. Data Encryption **Do This:** Use TLS encryption for all communication between your application and Redis, especially in cloud environments. Implement encryption at rest if sensitive data is stored. **Don't Do This:** Transmit sensitive data unencrypted over the network. Store unencrypted sensitive data in Redis. **Why:** Encryption protects data from eavesdropping and tampering during transit and storage. ## 7. Monitoring and Logging ### 7.1. Metrics Collection **Do This:** Collect key Redis metrics such as memory usage, CPU utilization, connection counts, and command latency. Use tools like Prometheus and Grafana for visualization. **Don't Do This:** Operate Redis without monitoring, making it difficult to identify performance bottlenecks or potential issues. **Why:** Monitoring provides valuable insights into the health and performance of your Redis deployment. ### 7.2. Structured Logging **Do This:** Implement structured logging with timestamps, log levels, and relevant context. Use consistent log formats to facilitate analysis and troubleshooting. **Don't Do This:** Rely on unstructured log messages, making them difficult to parse and analyze automatically. **Why:** Structured logs allow you to quickly identify and diagnose issues, track API usage, and monitor the overall health of your Redis integration.

# Component Design Standards for Redis This document outlines the component design standards for Redis development. It aims to provide guidelines for creating reusable, maintainable, and efficient components within the Redis ecosystem, leveraging modern approaches and design patterns. ## 1. Introduction Effective component design is crucial for the long-term health and scalability of Redis-based applications and modules. Well-designed components promote code reuse, reduce complexity, improve maintainability, and enhance overall system performance. These standards are tailored specifically to the nuances of Redis, considering its single-threaded nature, data structures, and performance constraints. ## 2. High-Level Component Design Principles These principles establish a foundational approach to component design in Redis. * **Single Responsibility Principle (SRP):** A component should have one, and only one, reason to change. This enhances modularity and reduces the risk of unintended side effects. * **Do This:** Design components that encapsulate a specific, well-defined task or responsibility. For example, a component for managing rate limiting, or one for handling user sessions. * **Don't Do This:** Create "god classes" or components that handle multiple unrelated responsibilities. This leads to tightly coupled code that is difficult to modify or test. * **Open/Closed Principle (OCP):** A component should be open for extension, but closed for modification. This promotes adding new functionality without changing existing code, thus reducing the risk of introducing bugs. * **Do This:** Use interfaces, abstract classes, and configuration to allow extending the component's behavior without modifying its core code. Consider Redis Modules' event system for extensions. * **Don't Do This:** Directly modify existing components to add new features. This violates the OCP and can lead to instability. * **Liskov Substitution Principle (LSP):** Subtypes must be substitutable for their base types. This ensures that derived classes can be used anywhere their base classes are used without causing errors. * **Do This:** Ensure that derived classes adhere to the contract defined by their base classes (interfaces or abstract classes). Return the same types, or subtypes, as the methods they override or implement in the base class. * **Don't Do This:** Create derived classes with methods that throw exceptions when called in contexts where the base class's methods would function normally. This violates the LSP and can lead to unexpected runtime errors. * **Interface Segregation Principle (ISP):** Clients should not be forced to depend upon interfaces that they do not use. This promotes smaller, more cohesive interfaces and reduces dependencies. * **Do This:** Create multiple, specific interfaces instead of large, monolithic ones. A component interacting with a cache should only depend on the interface for caching operations, not all possible database operations. * **Don't Do This:** Define huge interfaces that contain many methods, some of which some client components might not use. * **Dependency Inversion Principle (DIP):** Depend upon abstractions, not concretions. High-level modules should not depend on low-level modules. Both should depend on abstractions. Abstractions should not depend upon details. Details should depend upon abstrations. * **Do This:** Use dependency injection or service location to provide components with the dependencies they need. Always depend on interfaces or abstract classes rather than concrete implementations. * **Don't Do This:** Directly instantiate concrete dependencies within a component. This creates tight coupling and makes it difficult to test or replace dependencies. ## 3. Component Types in Redis Projects Different types of components are used within Redis projects and require different considerations. * **Data Access Components:** These components are responsible for interacting with Redis to retrieve, store, and manipulate data. * **Business Logic Components:** These components implement the core business logic of the application, often operating on data retrieved from Redis. * **Utility Components:** This encompasses reusable functions or classes that address generic tasks, such as serialization, logging, or configuration management. * **Redis Modules:** Components built in C/C++ to extend Redis core functionality. * **Lua Scripts:** Scripts executed server-side to implement complex operations. * **Pub/Sub Handlers:** Components for handling messages in the Redis publish/subscribe system. * **Streams Consumers:** Processes that read and process data from Redis Streams. ## 4. Coding Standards for Data Access Components Data Access Components handle all interactions to and from the Redis database. They can use Redis clients to read data, write data, and invalidate data. * **Abstraction of Redis Interaction:** Separate the business logic from the direct interactions with Redis by using dedicated data access components (e.g., repositories or DAOs). """python # Example: Data Access Component in Python using redis-py import redis class UserRepository: def __init__(self, redis_client): self.redis_client = redis_client def get_user(self, user_id): user_data = self.redis_client.hgetall(f"user:{user_id}") if user_data: return {k.decode(): v.decode() for k, v in user_data.items()} # Decode bytes return None def save_user(self, user_id, user_data): self.redis_client.hmset(f"user:{user_id}", user_data) def delete_user(self, user_id): self.redis_client.delete(f"user:{user_id}") # Example Usage: redis_client = redis.Redis(host='localhost', port=6379, db=0) user_repository = UserRepository(redis_client) user = user_repository.get_user("123") if user: print(f"User data: {user}") else: print("User not found") new_user = {"name": "John Doe", "email": "john.doe@example.com"} user_repository.save_user("456", new_user) """ * **Why:** Simplifies testing because you only need to mock the data access layer, not the redis client and ensures the service layer does not contain any database-specific code. * **Error Handling:** Implement robust error handling to catch Redis connection errors, timeouts, or other exceptions. """python import redis class UserRepository: def __init__(self, redis_client): self.redis_client = redis_client def get_user(self, user_id): try: user_data = self.redis_client.hgetall(f"user:{user_id}") if user_data: return {k.decode(): v.decode() for k, v in user_data.items()} return None except redis.exceptions.ConnectionError as e: print(f"Error connecting to Redis: {e}") return None except redis.exceptions.TimeoutError as e: print(f"Redis timeout error: {e}") return None except Exception as e: print(f"An unexpected error occurred: {e}") return None """ * **Why:** Improves the robustness and reliability of the application. Without proper error handling, Redis connection problems can cause the application to crash. * **Connection Pooling:** Use connection pooling to efficiently manage Redis connections and reduce overhead. Most Redis clients (e.g., "redis-py") offer built-in connection pooling. """python import redis # Using Connection Pool redis_pool = redis.ConnectionPool(host='localhost', port=6379, db=0, max_connections=10) redis_client = redis.Redis(connection_pool=redis_pool) def get_data(key): try: return redis_client.get(key) except redis.exceptions.ConnectionError as e: print(f"Connection Error: {e}") return None data = get_data("mykey") """ * **Why:** Reduces the overhead of creating new connections for each operation by reusing pre-existing connections. This saves time if multiple processes are writing to the same Redis instance. * **Serialization/Deserialization:** Use a consistent and efficient serialization format (e.g., JSON, MessagePack) for storing complex data structures in Redis. When retrieving data, ensure correct deserialization. """python import redis import json class UserRepository: def __init__(self, redis_client): self.redis_client = redis_client def get_user(self, user_id): user_json = self.redis_client.get(f"user:{user_id}") if user_json: try: return json.loads(user_json.decode()) except json.JSONDecodeError as e: print(f"JSONDecodeError: {e}") return None return None def save_user(self, user_id, user_data): user_json = json.dumps(user_data) self.redis_client.set(f"user:{user_id}", user_json) """ * **Why:** Improves data consistency and allows storing structured data in Redis. JSON (or MessagePack) can represent complex data in a readable, standardized serialized format. * **Key Naming Conventions:** Establish and adhere to a consistent key naming convention to improve data organization and prevent key collisions. Use namespaces and delimiters (e.g., ""user:{user_id}:name""). """python # Consistent Key Naming USER_NAMESPACE = "user" def get_user_key(user_id): return f"{USER_NAMESPACE}:{user_id}:data" # Example Usage: user_key = get_user_key("123") # Output: "user:123:data" redis_client.get(user_key) """ * **Why:** Makes the data model more organized and prevents keys with similar names from colliding. Using functions to build keys centralizes data access and makes key refactoring easier. ## 5. Coding Standards for Business Logic Components These components contain the core logic of the application and should be separated as much as possible from infrastructure and framework details. * **Dependency Injection:** Use dependency injection to provide business logic components with their necessary dependencies (including data access components). """python # Example using a simple dependency injection class UserService: def __init__(self, user_repository): self.user_repository = user_repository def get_user_name(self, user_id): user = self.user_repository.get_user(user_id) if user: return user.get("name") return None # Usage: user_service = UserService(UserRepository(redis_client)) # Injecting UserRepository user_name = user_service.get_user_name("123") """ * **Why:** Improves testability and modularity by decoupling components such as services and data. Dependency injection increases code readability by making dependencies very clear. * **Transaction Management:** When dealing with multiple Redis operations that need to be atomic, use Redis transactions ("MULTI", "EXEC", "DISCARD") or Lua scripts. Lua scripts provide better performance for complex operations. """python import redis def transfer_funds(redis_client, from_account, to_account, amount): try: with redis_client.pipeline() as pipe: pipe.watch(from_account, to_account) # Optimistic locking from_balance = int(pipe.get(from_account) or 0) to_balance = int(pipe.get(to_account) or 0) if from_balance < amount: return False # Insufficient funds pipe.multi() pipe.set(from_account, from_balance - amount) pipe.set(to_account, to_balance + amount) pipe.execute() return True except redis.WatchError: # Key was modified, retry the transaction or handle failure print("WatchError: Key modified during transaction. Retrying...") return False except redis.exceptions.ConnectionError as e: print(f"Redis connection error: {e}") return False # Example Usage redis_client = redis.Redis(host='localhost', port=6379, db=0) success = transfer_funds(redis_client, "account1", "account2", 50) if success: print("Funds transferred successfully.") else: print("Funds transfer failed.") """ * **Why:** Ensures that multiple related operations are executed as a single atomic unit, preventing data corruption, and data inconsistencies in race scenarios. "WATCH" provides optimistic locking. * **Validation:** Validate input data before processing to prevent errors and ensure data integrity. Validation can occur via validation libraries. """python from cerberus import Validator schema = { 'name': {'type': 'string', 'required': True}, 'age': {'type': 'integer', 'min': 0}, 'email': {'type':

# Security Best Practices Standards for Redis This document outlines security best practices for Redis development to help developers create secure and maintainable Redis-based applications. It provides actionable guidelines, code examples, and explanations to minimize vulnerabilities and ensure data security. ## 1. Authentication and Authorization ### 1.1. Require Authentication **Standard:** Always enable authentication on your Redis instances. **Do This:** Enable "requirepass" in redis.conf. **Don't Do This:** Never deploy a Redis instance without authentication, especially in production environments. **Why:** Unauthenticated Redis instances are vulnerable to unauthorized access, allowing attackers to read, modify, or delete data, and potentially execute arbitrary commands on the server. **Code Example:** """redis # redis.conf requirepass your_strong_password """ **Anti-Pattern:** Using default passwords or weak passwords. **Modern Approaches:** Utilize strong, randomly generated passwords stored securely and rotated regularly. Consider integrating with external authentication providers for centralized management. ### 1.2. Use ACLs (Access Control Lists) **Standard:** Use ACLs (introduced in Redis 6) to grant granular permissions to users. **Do This:** Create users with specific permissions based on their required access. **Don't Do This:** Rely solely on "requirepass" for all users. Avoid granting excessive privileges. **Why:** ACLs provide finer-grained control over who can access what data and execute which commands, reducing the risk of privilege escalation. **Code Example:** """redis ACL SETUSER user1 +get +set ~object:* ACL SETUSER user2 +info +client """ **Anti-Pattern:** Granting "ALL COMMANDS" to all users. **Modern Approaches:** Leverage ACLs to implement the principle of least privilege. Regularly review and adjust permissions as needed. Use ACL categories (e.g., "@read", "@write", "@admin") to simplify permission management. ### 1.3 Restrict "RENAME" Command **Standard:** Carefully control access to the "RENAME" command, especially in shared environments. **Do This:** Deny "RENAME" access to users who don't need it. **Don't Do This:** Allow unrestricted use of "RENAME" on sensitive keys as this could lead to data corruption or information disclosure. **Why:** "RENAME" can overwrite existing keys, potentially causing data loss or security vulnerabilities. **Code Example:** """redis ACL SETUSER user3 -rename """ ### 1.4 Secure Redis Cluster **Standard:** Enable authentication and authorization across all nodes in a Redis Cluster. **Do This:** Configure "requirepass" and ACLs consistently on every node. Use the "CLUSTER" command appropriately, ensuring nodes are communicating securely. **Don't Do This:** Assume security on one node translates to security across the cluster. Ignoring security within cluster communication. **Why:** A compromised node in a cluster can compromise the entire dataset. Secure internal cluster communication prevents rogue nodes from joining and extracting information. ## 2. Network Security ### 2.1. Bind to Specific Interfaces **Standard:** Bind Redis to specific network interfaces, restricting access from outside the local network. **Do This:** Configure "bind" in redis.conf to listen only on specific IPs (e.g., "127.0.0.1" for local access, or a specific internal network IP). **Don't Do This:** Bind Redis to "0.0.0.0" (all interfaces) in production without proper firewall protection. **Why:** Binding to all interfaces exposes Redis to potential attacks from external networks. **Code Example:** """redis # redis.conf bind 127.0.0.1 10.0.0.10 """ **Anti-Pattern:** Binding to "0.0.0.0" without implementing a strong firewall. **Modern Approaches:** Employ a combination of interface binding, firewall rules, and network segmentation to restrict access. ### 2.2. Use Firewalls **Standard:** Implement firewall rules to allow only necessary traffic to Redis instances. **Do This:** Configure firewalls (e.g., iptables, ufw, cloud provider firewalls) to allow only trusted IP addresses and ports to communicate with Redis. **Don't Do This:** Rely solely on Redis authentication without a firewall. **Why:** Firewalls provide an additional layer of security, preventing unauthorized network access to Redis. **Code Example (iptables):** """bash iptables -A INPUT -s 10.0.0.0/24 -p tcp --dport 6379 -j ACCEPT iptables -A INPUT -p tcp --dport 6379 -j DROP """ **Modern Approaches:** Use cloud-native firewall solutions and network policies to manage network access dynamically. ### 2.3. Enable TLS Encryption **Standard:** Enable TLS (Transport Layer Security) encryption for client-server and cluster communication. This is supported natively in Redis 6 and later. **Do This:** Configure "tls-port", "tls-cert-file", "tls-key-file", and "tls-ca-cert-file" in redis.conf. **Don't Do This:** Use Redis over plain TCP in production environments, especially when transmitting sensitive data. **Why:** TLS encryption protects data in transit, preventing eavesdropping and man-in-the-middle attacks. **Code Example:** """redis # redis.conf tls-port 6380 tls-cert-file /path/to/redis.crt tls-key-file /path/to/redis.key tls-ca-cert-file /path/to/ca.crt """ **Modern Approaches:** Use automated certificate management tools (e.g., Let's Encrypt, HashiCorp Vault) to simplify certificate provisioning and rotation. Utilize mutual TLS (mTLS) for enhanced security, requiring clients to authenticate with certificates. ## 3. Input Validation and Sanitization ### 3.1. Sanitize User Input **Standard:** Sanitize any user-provided input before using it in Redis commands. **Do This:** Use appropriate escaping or encoding techniques to prevent command injection vulnerabilities. **Don't Do This:** Directly concatenate user input into Redis commands without sanitization. **Why:** Without proper sanitization, attackers can inject malicious commands into Redis. **Code Example (Python with redis-py):** """python import redis import redis.utils r = redis.Redis(host='localhost', port=6379, password='your_password') user_key = 'user:' + redis.utils.escape_string(user_id) # Sanitize user_id r.set(user_key, user_data) """ **Anti-Pattern:** Using Python's "str.format" or f-strings without escaping when constructing Redis commands. **Modern Approaches:** Leverage Redis client libraries that provide built-in sanitization and parameterization features. Consider using prepared statements where applicable (although Redis doesn't natively support them in the traditional SQL sense, client libraries often provide similar functionality). ### 3.2. Limit Command and Key Lengths **Standard:** Enforce reasonable limits on the length of keys and values to prevent denial-of-service attacks and memory exhaustion. **Do This:** Implement validation logic in your application to reject overly long keys or values. **Don't Do This:** Allow unbounded key lengths, which could lead to performance degradation or Redis crashes. **Why:** Extremely long keys or values can consume excessive memory and CPU resources. ## 4. Configuration and Operational Security ### 4.1. Disable Dangerous Commands **Standard:** Disable or rename potentially dangerous commands (e.g., "FLUSHALL", "FLUSHDB", "KEYS", "EVAL") in redis.conf. **Do This:** Use the "rename-command" directive in redis.conf to either disable or rename these commands. **Don't Do This:** Leave dangerous commands enabled in production environments. **Why:** These commands can be abused to delete data, expose sensitive information, or execute arbitrary Lua scripts. **Code Example:** """redis # redis.conf rename-command FLUSHALL "" rename-command FLUSHDB "" rename-command KEYS very_secret_key_enumeration_command rename-command EVAL "" """ **Modern Approaches:** Use ACLs in conjunction with command renaming for a more granular level of control. ### 4.2. Limit Memory Usage **Standard:** Configure "maxmemory" in redis.conf to limit the amount of memory Redis can use. Set an appropriate "maxmemory-policy" (e.g., "volatile-lru", "allkeys-lru") to evict keys when the memory limit is reached. **Do This:** Set "maxmemory" and choose a suitable "maxmemory-policy" based on your application's needs. **Don't Do This:** Allow Redis to consume unlimited memory, which can lead to out-of-memory errors and system instability. **Why:** Limiting memory usage prevents Redis from consuming all available system memory, ensuring stability and preventing denial-of-service conditions. **Code Example:** """redis # redis.conf maxmemory 2gb maxmemory-policy allkeys-lru """ ### 4.3. Secure Lua Scripting **Standard:** Carefully review and audit Lua scripts used with the "EVAL" command. Parameterize input to avoid injection vulnerabilities. Disable the "EVAL" command entirely if not needed. **Do This:** Validate and sanitize any user-provided input used within Lua scripts. Use "EVALSHA" to execute pre-compiled scripts. **Don't Do This:** Concatenate user input directly into Lua scripts. **Why:** Lua scripts can execute arbitrary code within the Redis server, so vulnerabilities in scripts can lead to severe security breaches. **Code Example:** """python import redis r = redis.Redis(host='localhost', port=6379, password='your_password') script = """ local key = KEYS[1] local value = ARGV[1] redis.call('SET', key, value) return redis.call('GET', key) """ sha = r.script_load(script) # Load the script result = r.evalsha(sha, 1, 'mykey', 'myvalue') # use EVALSHA for security print(result) """ **Modern Approaches:** Use server-side scripting with caution. Consider alternatives, such as Redis Modules, for more complex functionality with stricter security controls. ### 4.4 Regular Security Audits and Updates **Standard:** Regularly audit your Redis configuration, code, and dependencies for security vulnerabilities. Keep Redis updated to the latest stable version. **Do This:** Subscribe to security advisories and apply patches promptly. Perform regular security scans using vulnerability assessment tools. **Don't Do This:** Run outdated versions of Redis without applying security patches. **Why:** Regular audits and updates help identify and address security vulnerabilities before they can be exploited. ## 5. Data Encryption at Rest (Redis Enterprise and Modules) **Standard:** For sensitive data, consider using data encryption at rest. This is not natively supported in open-source Redis but is a standard feature of Redis Enterprise. Modules like RedisCrypt also provide encryption. **Do This:** If using Redis Enterprise, configure data encryption at rest. If using open-source Redis, evaluate and implement a suitable encryption module. **Don't Do This:** Store sensitive data in plain text without encryption. **Why:** Encryption at rest protects data from unauthorized access if the storage media is compromised. **Modern Approaches:** Use key management systems (KMS) to securely store and manage encryption keys. ## 6. Monitoring and Logging ### 6.1. Enable Comprehensive Logging **Standard:** Configure Redis to log all important events, including connection attempts, authentication failures, and command execution. **Do This:** Set the "loglevel" configuration option in redis.conf to "verbose" or "debug". Configure syslog integration for centralized logging. **Don't Do This:** Disable logging, or set the log level too low, which can hinder security investigations. **Why:** Comprehensive logging provides valuable information for security monitoring, incident response, and auditing. **Code Example:** """redis # redis.conf loglevel verbose """ ### 6.2. Monitor Redis Performance and Security **Standard:** Monitor Redis performance metrics (e.g., CPU usage, memory usage, connection count, slow queries) to detect anomalies that may indicate a security breach. Monitor security-related events (e.g., authentication failures, unauthorized command execution). **Do This:** Use tools like RedisInsight, Prometheus, Grafana, or cloud provider monitoring services to track Redis metrics. Set up alerts for suspicious activity. **Don't Do This:** Ignore Redis performance and security metrics. **Why:** Proactive monitoring helps detect and respond to security incidents in a timely manner. ## 7. Connection Pooling ### 7.1 Properly Configure Connection Pools **Standard:** Use and properly configure connection pools in your application to optimize resource utilization and prevent connection leaks. **Do This:** Ensure that the connection pool has appropriate limits for maximum and minimum connections, timeout settings, and connection health checks. **Don't Do This:** Create a new connection to Redis for every operation, as this can be resource-intensive and lead to performance bottlenecks. **Why:** Efficient connection pooling minimizes the overhead of establishing and tearing down connections, improving performance and stability. **Code Example (Python with redis-py):** """python import redis pool = redis.ConnectionPool(host='localhost', port=6379, db=0, password='your_password', max_connections=100) r = redis.Redis(connection_pool=pool) # Use the redis client 'r' as needed """ ### 7.2. Secure Connection Credentials **Standard:** Securely store and manage connection credentials, such as passwords, to prevent unauthorized access. **Do This:** Store passwords in environment variables, configuration files with restricted access, or secrets management systems (e.g., HashiCorp Vault). **Don't Do This:** Hardcode credentials directly in the application code. **Why:** Storing credentials securely prevents them from being exposed to unauthorized users. ## 8. Understand Redis Vulnerabilities ### 8.1 Stay Informed. **Standard:** Keep abreast of common Redis Vulnerabilities **Do This:** Monitor alerts from the Redis project, vulnerability databases (e.g. CVE), and cloud providers if hosting a managed instance. **Don't Do This:** Assume your current configuration is secure forever. New vulnerabilities are discovered constantly. **Why:** Knowing what to look for will improve code quality, configuration, and detection of malicious activity. ### 8.2 Protect Against Known Vulnerabilities **Standard:** Address known Redis vulnerabilities – especially around Lua scripting, command injection and default configurations. **Do This:** Follow recommendations from official Redis documentation on patching and mitigation. **Don't Do This:** Delay the application of security patches and recommended configuration changes, as this leaves systems vulnerable to exploitation. ## 9. Client-Side Security ### 9.1 Validate Data Retrieved from Cache **Standard:** Validate cache data on the client side **Do This:** Implement checks to ensure data is the expected type, within reasonable bounds, and matches with original schemas. **Don't Do This:** Blindly trust data pulled from the Redis cache. **Why:** Prevents vulnerabilities due to data corruption in the cache, particularly if external processes also have write access to Redis. ### 9.2 Secure Client Library Usage **Standard:** Use official, maintained client libraries and keep them up to date. **Do This:** Regularly update client libraries to benefit from latest bug fixes and performance improvements, and security patches. Pin library versions in deployment configurations. **Don't Do This:** Build your own client library. Use deprecated or unmaintained libraries. **Why:** Well vetted client libraries are less vulnerable and more performant than custom implementations.