Cline

# Code Style and Conventions Standards for Redis This document outlines the coding style and conventions to be followed when contributing to the Redis project. Adhering to these guidelines ensures code readability, maintainability, and consistency across the codebase. These conventions are intended to be used by both human contributors and AI-powered coding assistants to maintain a high standard of quality. This document assumes familiarity with C programming, data structures, and basic Redis concepts. ## 1. Formatting Consistent formatting is crucial for readability. We primarily follow the style of the existing Redis codebase. ### 1.1. Indentation * **Do This:** Use 4 spaces for indentation. Tabs are strictly forbidden. * **Don't Do This:** Use tabs or inconsistent indentation. * **Why:** Spaces provide consistent rendering across different editors and platforms. """c // Correct indentation int my_function(int arg1, int arg2) { if (arg1 > arg2) { return arg1 - arg2; } else { return arg2 - arg1; } } // Incorrect indentation (tabs) int my_function(int arg1, int arg2) { \tif (arg1 > arg2) { \t\treturn arg1 - arg2; \t} else { \t\treturn arg2 - arg1; \t} } // Incorrect indentation (inconsistent) int my_function(int arg1, int arg2) { if (arg1 > arg2) { return arg1 - arg2; } else { return arg2 - arg1; } } """ ### 1.2. Line Length * **Do This:** Limit lines to a maximum of 80 characters. Break long lines appropriately. * **Don't Do This:** Allow lines to exceed 80 characters unless absolutely necessary (e.g., very long string literals). * **Why:** Limits improve readability on smaller screens and when viewing diffs. """c // Correct line breaking void very_long_function_name( int very_long_argument_name1, int very_long_argument_name2) { // ... code ... } // Incorrect line length void very_long_function_name(int very_long_argument_name1, int very_long_argument_name2) { // ... code ... } """ ### 1.3. Spaces * **Do This:** * Use spaces around operators (=, +, -, *, /, %, ==, !=, >, <, >=, <=, &&, ||, etc.). * Use spaces after commas in argument lists. * Use a space after "if", "for", "while", and "switch" keywords. * No space between function name and parentheses. * **Don't Do This:** Omit spaces around operators or after commas, or include spaces between function names and parentheses. * **Why:** Clear spacing enhances readability. """c // Correct use of spaces int result = a + b * (c - d); my_function(arg1, arg2, arg3); if (x > 0) { /* ... */ } // Incorrect use of spaces int result=a+b*(c-d); my_function(arg1,arg2,arg3); if(x>0){ /* ... */ } """ ### 1.4. Braces * **Do This:** Always use braces for "if", "else", "for", "while", and "do-while" statements, even for single-line bodies. Place the opening brace on the same line as the statement. * **Don't Do This:** Omit braces for single-line bodies, or place the opening brace on a new line. * **Why:** Braces prevent errors during modification and improve code consistency. """c // Correct brace usage if (condition) { do_something(); } else { do_something_else(); } for (int i = 0; i < 10; i++) { process_data(i); } // Incorrect brace usage if (condition) do_something(); //This is wrong and discouraged if(condition){ // Also wrong. do_something(); } """ ### 1.5. Vertical Spacing * **Do This:** Use blank lines to separate logical blocks of code within a function. Separate function definitions with blank lines. * **Don't Do This:** Compress code too tightly or leave excessive blank lines. * **Why:** Enhanced readability. """c // Correct Vertical Spacing int process_data(int data) { // Initialization int result = 0; // Processing logic if (data > 0) { result = data * 2; } else { result = data / 2; } return result; } int main() { int value = 10; int processed_value = process_data(value); printf("Processed value: %d\n", processed_value); return 0; } """ ## 2. Naming Conventions Clear and consistent naming is essential for understanding code. ### 2.1. General Naming * **Do This:** Use descriptive names that clearly indicate the purpose of variables, functions, and constants. * **Don't Do This:** Use single-letter variable names (except for loop counters like "i", "j", "k"), cryptic abbreviations, or names that are misleading. * **Why:** Improves readability and comprehension. ### 2.2. Variables * **Do This:** Use "snake_case" for variable names. * **Don't Do This:** Use "camelCase", "PascalCase", or uppercase. * **Why:** Consistent with the existing codebase and common C practice. """c // Correct variable naming int user_id; char *user_name; size_t data_length; // Incorrect variable naming int userID; // camelCase int UserId; // PascalCase int USER_ID; // Uppercase (usually for constants) int x; // Undescriptive """ ### 2.3. Functions * **Do This:** Use "snake_case" for function names. Functions that are internally used (private) within a module or source file should be prefixed with "module_". * **Don't Do This:** Use "camelCase", "PascalCase", or uppercase function names. Do not use global functions unless absolutely necessary. * **Why:** Consistent with the coding style. Module prefix avoids name collisions and clearly indicates function scope. """c // Correct function naming void process_input_data(char *input); int calculate_average(int *data, int count); void module_internal_function(void); //File-scope internal utility function. // Incorrect function naming void ProcessInputData(char *input); // PascalCase void processInputData(char *input); // camelCase void PROCESS_INPUT(char *input); // Uppercase """ ### 2.4. Constants and Macros * **Do This:** Use uppercase with "snake_case" for constants and macros. * **Don't Do This:** Use lowercase or camelCase for constants and macros. * **Why:** Convention to distinguish constants and macros from variables. """c // Correct constant naming #define MAX_CONNECTIONS 1024 const int DEFAULT_PORT = 6379; // Incorrect constant naming #define maxConnections 1024 const int defaultPort = 6379; """ ### 2.5. Data Structures (structs, enums, unions) * **Do This:** Use "PascalCase" for struct, enum, and union names. * **Don't Do This:** Use "snake_case" or lowercase. * **Why:** Consistent with common C practices for type definitions. """c // Correct data structure naming typedef struct UserProfile { int user_id; char *user_name; } UserProfile; typedef enum ConnectionState { CONNECTED, DISCONNECTED, CONNECTING } ConnectionState; // Incorrect data structure naming typedef struct user_profile { int user_id; char *user_name; } user_profile; """ ## 3. Stylistic Consistency Maintaining a consistent style enhances readability and reduces cognitive load. ### 3.1. Comments * **Do This:** * Write clear and concise comments to explain complex logic, algorithms, and data structures. * Use block comments ("/* ... */") for multi-line comments and function headers. * Use single-line comments ("// ...") for brief explanations within a function. * Document the purpose, arguments, and return value of each function. * **Don't Do This:** * Write unnecessary or redundant comments that simply repeat the code. * Use cryptic or ambiguous comments. * Fail to update comments when the code changes. * **Why:** Comments provide context and help others understand the code. """c // Correct comment usage /* * This function calculates the average of an array of integers. * * Parameters: * data: A pointer to the array of integers. * count: The number of elements in the array. * * Returns: * The average of the integers, or 0 if the array is empty. */ int calculate_average(int *data, int count) { if (count == 0) { return 0; // Return 0 for an empty array. } int sum = 0; for (int i = 0; i < count; i++) { sum += data[i]; } return sum / count; } // Incorrect comment usage int calculate_average(int *data, int count) { if (count == 0) { return 0; } int sum = 0; for (int i = 0; i < count; i++) { sum += data[i]; } return sum / count; // Returns the average } """ ### 3.2. Error Handling * **Do This:** * Check for errors and handle them gracefully. Use return values or error codes to indicate success or failure. * Log error messages with sufficient context to aid in debugging. * Clean up resources (memory, file descriptors, etc.) before returning from an error condition. * **Don't Do This:** * Ignore errors or assume that functions always succeed. * Print error messages to stdout without context. * Leak resources on error. * **Why:** Robust error handling is crucial for stability and reliability. """c // Correct Error Handling #include <stdio.h> #include <stdlib.h> int allocate_memory(size_t size, void **ptr) { *ptr = malloc(size); if (*ptr == NULL) { perror("Failed to allocate memory"); // Correct error message return -1; // Correct return value for error } return 0; // Correct return value for success } int main() { void *data; if (allocate_memory(1024, &data) != 0) { fprintf(stderr, "Memory allocation failed.\n"); // Proper error reporting goes to stderr. return 1; // Use non-zero return code to signal failure. } // ... use data ... free(data); return 0; } // Incorrect error handling #include <stdio.h> #include <stdlib.h> int allocate_memory(size_t size, void **ptr) { *ptr = malloc(size); return 0; // Always return success, even if malloc fails } int main() { void *data; allocate_memory(1024, &data); // No error checking // ... use data (potential crash if malloc failed) ... free(data); return 0; } """ ### 3.3. Memory Management * **Do This:** * Allocate and free memory carefully, using "malloc", "calloc", "realloc", and "free". * Always free memory when it is no longer needed to prevent memory leaks. * Use "valgrind" or similar tools to detect memory leaks and errors. * **Don't Do This:** * Forget to free allocated memory. * Free the same memory multiple times (double-free). * Use memory after it has been freed (use-after-free). * **Why:** Proper memory management is essential for preventing crashes and performance degradation. Redis, being a performance-critical application, requires meticulous memory handling. """c // Correct memory management #include <stdlib.h> char *duplicate_string(const char *str) { size_t length = strlen(str) + 1; char *duplicate = (char *)malloc(length); if (duplicate == NULL) { return NULL; // Handle allocation failure } strcpy(duplicate, str); return duplicate; } int main() { char *original = "Redis"; char *copy = duplicate_string(original); if (copy != NULL) { printf("Copy: %s\n", copy); free(copy); // Correctly free the allocated memory } return 0; } // Incorrect memory management #include <stdlib.h> char *duplicate_string(const char *str) { size_t length = strlen(str) + 1; char *duplicate = (char *)malloc(length); strcpy(duplicate, str); return duplicate; } int main() { char *original = "Redis"; char *copy = duplicate_string(original); // Memory allocated for 'copy' is never freed (memory leak) return 0; } """ ### 3.4. Data Structures * **Do This**: Use appropriate data structures for the task at hand. Redis makes extensive uses of "sds" strings, linked lists via "list", dictionaries via "dict", and skiplists. Favor these native data structures where appropriate for optimized performance. * **Don't Do This**: Reimplement existing data structures or use less efficient options. * **Why**: Leverages optimized, well-tested components, contributing to performance and stability. """c // Correct use of SDS strings in Redis #include "sds.h" //Include the sds implementation. int main() { sds my_string = sdsnew("Hello, Redis!"); printf("SDS string: %s\n", my_string); my_string = sdscat(my_string, " - Updated"); // Append to the string safely printf("Updated SDS string: %s\n", my_string); sdsfree(my_string); // Free the SDS string return 0; } // Incorrect (less efficient) use of standard char * #include <stdio.h> #include <stdlib.h> #include <string.h> int main() { char *my_string = malloc(strlen("Hello, Redis!") + 1); strcpy(my_string, "Hello, Redis!"); printf("C string: %s\n", my_string); // Appending requires manual memory management and is prone to errors char *temp = realloc(my_string, strlen(my_string) + strlen(" - Updated") + 1); if (temp == NULL) { free(my_string); // Prevent memory leak on realloc failure. return 1; } my_string = temp; strcat(my_string, " - Updated"); printf("Updated C string: %s\n", my_string); free(my_string); // Free the C string return 0; } """ ## 4. Redis-Specific Considerations These are styles and conventions that are specific to core Redis server development. ### 4.1. Event Loop Integration * **Do This:** Integrate new functionality smoothly with the Redis event loop. Use "aeCreateFileEvent" to register file descriptors for reading and writing events. Avoid blocking operations; prefer asynchronous models. * **Don't Do This:** Introduce blocking calls that can halt the entire server. * **Why:** Redis's performance relies heavily on its non-blocking event loop. """c // Example: Adding a new file event to the Redis event loop (simplified) #include "ae.h" //Include the ae implementation void my_read_handler(aeEventLoop *loop, int fd, void *clientData, int mask) { // Handle data received on the file descriptor printf("Data received on fd %d\n", fd); } int initialize_my_feature(aeEventLoop *loop, int fd) { // Create a file event to listen for readability on the file descriptor if (aeCreateFileEvent(loop, fd, AE_READABLE, my_read_handler, NULL) == AE_ERR) { fprintf(stderr, "Error creating file event\n"); return -1; } return 0; } """ ### 4.2. Command Implementation * **Do This**: Redis commands should be implemented as standalone functions that take a "client" struct as their primary argument. Follow the existing command structure for argument parsing, validation, and reply formatting. Use "addReply*" functions to construct the response. Careful adherence to the existing command structure is extremely important. * **Don't Do This**: Hardcode command logic directly into the event loop or create commands with inconsistent argument handling. * **Why:** Ensures commands are easily manageable, testable, and well-integrated with the Redis protocol. """c // Example: Implementing a simple Redis command (simplified) #include "server.h" void mycommandCommand(client *c) { // Check the number of arguments. Example will require zero arguments if (c->argc != 1) { addReplyErrorArity(c); return; } // Command logic - in this example simply returns "OK" if the call succeeds addReplyString(c, "OK"); //Reply with a simple string result and no error } //Add to the commmand table inside populateCommandTable() using the following pattern: // {"mycommand",mycommandCommand,-1,"readonly",0,NULL,0,0,0,0,0} """ ### 4.3. Configuration Handling * **Do This:** Use the existing configuration framework in "server.c" to manage new configuration options. Define appropriate defaults and validation logic for each option. * **Don't Do This:** Introduce hardcoded configuration values or bypass the configuration system. * **Why:** Centralized configuration management provides consistency and simplifies administration. ### 4.4. Logging * **Do This**: Utilize the "serverLog()" function for all logging within Redis. Choose the appropriate log level (e.g., "LL_DEBUG", "LL_VERBOSE", "LL_WARNING", "LL_NOTICE", "LL_ERROR") based on the severity of the message. * **Don't Do This**: Use "printf()" directly, or choose inappropriate log levels. * **Why**: Enables consistent logging, filtering, and analysis. """c // Example logging with serverLog #include "server.h" // Include the server declarations void my_function(int value) { if (value < 0) { serverLog(LL_WARNING, "Received negative value: %d", value); return; } serverLog(LL_DEBUG, "Processing value: %d", value); // ... rest of the function } """ ### 4.5. Data Type Handling * **Do This:** Use the built-in Redis data type abstractions (e.g., "robj") for storing and manipulating data. These abstractions handle memory management, encoding, and other details. * **Don't Do This:** Directly manipulate raw memory or bypass the data type system. * **Why**. Ensures consistent and efficient data handling throughout the server. """c //Correct use of redis objects. #include "server.h" void my_function(redisDb *db, robj *key, robj *value){ //Value will depend on what is being passed in in key and value, // but it will always be a "robj" data type. dictAdd(db->dict, key, value); //Example function. } """ ## 5. Security Best Practices Security is paramount in Redis development. ### 5.1. Input Validation * **Do This:** Thoroughly validate all input data to prevent buffer overflows, format string vulnerabilities, and other security issues. Use safe string handling functions like "sdscat" and "sdscats". * **Don't Do This:** Trust input data without validation, or use unsafe string functions like "strcpy". * **Why:** Input validation is crucial for preventing security exploits. ### 5.2. Privilege Separation * **Do This:** Run Redis with the lowest possible privileges. Use a dedicated user account for Redis and disable unnecessary features. * **Don't Do This:** Run Redis as root or with excessive privileges. * **Why:** Limits the impact of potential security breaches. ### 5.3. Avoid Deprecated Features * **Do This:** AVOID using deprecated or known vulnerable features unless absolutely necessary. Always check the release notes for security vulnerabilities. * **Don't Do This:** Use deprecated functions that have been replaced with safer alternatives. * **Why:** Using up-to-date code will help prevent security flaws. ### 5.4. Code Review * **Do This:** Have your code reviewed by other experienced Redis developers before committing changes. Code review is a crucial step in identifying potential security vulnerabilities and other issues. * **Don't Do This:** Push code without review. * **Why** Increased security and code confidence. ## 6. Testing * **Do This**: Write comprehensive unit tests and integration tests to verify the correctness and performance of new code. Use the existing Redis testing framework. * **Don't Do This**: Neglect testing, or rely solely on manual testing. * **Why**: Automated testing ensures code quality and prevents regressions.

# Core Architecture Standards for Redis This document outlines the core architectural standards for Redis development. It aims to guide developers in building maintainable, performant, and secure Redis applications. These standards reflect modern best practices based on the latest version of Redis (at the time of writing, this is considered to be Redis 7.x, though upcoming versions will have their considerations). The focus is to establish a consistent approach to project structure, architectural patterns, and implementation details, enabling a cohesive and high-quality codebase. ## 1. Project Structure and Organization A well-defined project structure promotes code discoverability, testability, and maintainability. ### 1.1. Standard Directory Layout **Do This:** Adhere to a consistent directory structure, separating concerns. """ redis-module/ ├── src/ # Source code files: C code, API implementations ├── include/ # Header files: Module definitions, API declarations ├── tests/ # Test suite: Unit tests, integration tests ├── deps/ # Third-party dependencies (if any, highly discouraged to modify) ├── Makefile # Build instructions ├── README.md # Project description and setup instructions ├── LICENSE # Licensing information └── config.mk # Configuration parameters for compilation """ **Don't Do This:** Intermix source code, tests, and build scripts in a single directory. **Why:** Clean separation of concerns simplifies navigation, compilation, and testing. Modifications to tests or build configurations are less likely to unintentionally affect the core source code. **Code Example:** """makefile # Example Makefile excerpt CFLAGS = -Wall -O2 -g -I./include -I./deps/hiredis LDFLAGS = -shared redismodule.so: src/*.c $(CC) $(CFLAGS) -fPIC $^ -o $@ $(LDFLAGS) test: redismodule.so tests/*.c $(CC) $(CFLAGS) -I./include -L. -lredismodule $^ -o test_runner ./test_runner """ ### 1.2. Module Decomposition **Do This:** Decompose large modules into smaller, functionally cohesive units. **Don't Do This:** Create monolithic modules containing unrelated functionality. **Why:** Smaller modules are easier to understand, test, and reuse. This improves code maintainability and reduces the likelihood of introducing bugs during modifications. **Code Example:** Imagine a module with extensive string manipulation functions. Break it down: """c // string_utils.h #ifndef STRING_UTILS_H #define STRING_UTILS_H #include <stddef.h> char *string_duplicate(const char *str); size_t string_length(const char *str); // ... other string utility function declarations #endif """ """c // string_utils.c #include "string_utils.h" #include <stdlib.h> #include <string.h> char *string_duplicate(const char *str) { size_t len = string_length(str); char *new_str = malloc(len + 1); if (new_str == NULL) return NULL; // Handle allocation failure memcpy(new_str, str, len + 1); return new_str; } size_t string_length(const char *str) { return strlen(str); } // ... other string utility function implementations """ This separation allows "string_utils" to be tested independently and reused in other parts of your module or even different modules. ## 2. Architectural Patterns Choosing appropriate architectural patterns is crucial for scalability, maintainability, and performance. ### 2.1. Event-Driven Architecture **Do This:** Leverage Redis's Pub/Sub functionality for asynchronous communication between services. **Don't Do This:** Rely solely on synchronous request-response patterns when asynchronous alternatives exist. **Why:** Event-driven architectures decouple services, improve responsiveness, and enable scalability. Pub/Sub allows services to react to events without direct dependencies on the event source. **Code Example:** Using Redis Pub/Sub with a simple publisher and subscriber. """python # Publisher (Python example using redis-py) import redis import time r = redis.Redis(host='localhost', port=6379, db=0) for i in range(10): r.publish('my_channel', f'Message {i}') print(f"Published: Message {i}") time.sleep(1) """ """python # Subscriber (Python example using redis-py) import redis r = redis.Redis(host='localhost', port=6379, db=0) pubsub = r.pubsub() pubsub.subscribe('my_channel') for message in pubsub.listen(): if message['type'] == 'message': print(f"Received: {message['data'].decode('utf-8')}") """ ### 2.2. Data Locality **Do This:** Design your data model to maximize data locality within Redis. Use appropriate data structures to keep related data together. **Don't Do This:** Spread related data across multiple keys unnecessarily, leading to increased network latency and lookup overhead. **Why:** Data locality improves performance by reducing the number of round trips to the Redis server. Using appropriate data structures (e.g., Hashes, Lists, Sets, Sorted Sets, Streams) allows you to group related data within a single key. **Code Example:** Storing user profile data in a Hash instead of individual keys. """redis # Anti-pattern: Storing user data in separate keys SET user:123:name "John Doe" SET user:123:email "john.doe@example.com" SET user:123:age 30 # Best practice: Storing user data in a Hash HSET user:123 name "John Doe" email "john.doe@example.com" age 30 # Retrieving all user data with a single command HGETALL user:123 """ Maintaining data locality minimizes the network round trips. For more complex relationships consider RedisJSON and RedisGraph modules. ### 2.3. Caching Strategies **Do This:** Implement caching strategies appropriate for your application's needs (e.g., write-through, write-back, cache-aside). Carefully consider TTLs and cache invalidation policies. **Don't Do This:** Use Redis solely as a persistent data store without leveraging its caching capabilities. Neglect to implement proper cache invalidation, leading to stale data. **Why:** Redis is primarily an in-memory data store optimized for caching. Leveraging caching strategies improves performance and reduces the load on persistent databases. **Code Example:** Implementing a cache-aside pattern in Python. """python import redis import json import time r = redis.Redis(host='localhost', port=6379, db=0) def get_user_profile(user_id, get_from_db): # get_from_db is a function to fetch from DB cache_key = f"user:{user_id}:profile" cached_data = r.get(cache_key) if cached_data: print("Fetching from cache") return json.loads(cached_data.decode('utf-8')) # Decode bytes print("Fetching from database") user_profile = get_from_db(user_id) # Assuming a function to fetch from DB r.setex(cache_key, 3600, json.dumps(user_profile)) #setex for TTL return user_profile # Example Usage(replace get_user_from_db with your function): # assuming you have a function to fetch the profile from the db # def get_user_profile_from_db(user_id): # # DB access code here using SQL or similar to fetch the profile # return user_profile_data #profile = get_user_profile(123, get_user_profile_from_db) #print(profile) """ ### 2.4 Redis Streams for Eventing **Do This:** Use Redis Streams for reliable, persistent event logging and complex event processing. Groups and consumer management within Streams avoid message loss. **Don't Do This:** Rely solely on Pub/Sub for asynchronous eventing where message delivery guarantees are required. Overcomplicate fan-out patterns with manual management when groups can handle it. **Why:** Streams provides at-least-once delivery, persistence, consumer groups, and the ability for clients to consume messages in a reliable fashion. This functionality is ideal for event sourcing, change data capture (CDC), and queueing applications. **Code Example**: Reading a message from a consumer group: """python import redis redis_client = redis.Redis(host='localhost', port=6379) stream_key = 'my_stream' group_name = 'my_group' consumer_name = 'consumer_1' # Create stream group try: redis_client.xgroup_create(stream_key, group_name, id='0') except redis.exceptions.ResponseError as e: if str(e) == 'BUSYGROUP Consumer Group name already exists': print("Consumer group already exists") else: raise e """ """python # Consume messages from the group while True: response = redis_client.xreadgroup(groupname=group_name, consumername=consumer_name, streams={stream_key: '>'}, count=1, block=5000) #Block indefinitely for a new message if response: stream_name, messages = response[0] message_id, message_data = messages[0] print(f"Received message: {message_data}") redis_client.xack(stream_key, group_name, message_id) # Acknowledge the message else: print("No new messages.") """ ### 2.5 Data Modeling in Redis. **Do This**: Choose the appropriate data structure considering access patterns, data size, and operations. **Don't Do This**: Using only STRINGS for everything. **Why**: Redis offers multiple data structures engineered for speed based on the specific operations they optimize. **Code Example**: * **Strings**: Simple key-value storage. * **Lists**: Ordered collections, suited for queues or time series. * **Sets**: Unique, unordered collections, great for membership testing. * **Hashes**: Record-like structures within a key. * **Sorted Sets**: Ordered sets with scoring, perfect for leaderboards and range queries. * **Streams:** Append-only log; messaging and event sourcing. * **Geospatial Indexes**: Store and query geographic data. * **Bitmaps & HyperLogLogs**: Space-efficient data structures for specific tasks. Example using a ZSET for a leaderboard: """redis ZADD leaderboard 100 "player1" ZADD leaderboard 150 "player2" ZADD leaderboard 120 "player3" ZRANGE leaderboard 0 -1 REV # Fetch the leaderboard in descending order """ ## 3. Implementation Details Adhering to consistent coding conventions and best practices improves code readability, maintainability, and collaboration. ### 3.1. Error Handling **Do This:** Implement robust error handling. Check return values of Redis commands and handle potential errors gracefully. **Don't Do This:** Ignore error conditions, leading to unpredictable behavior and potential data corruption. **Why:** Proper error handling prevents application crashes and ensures data integrity. Log errors appropriately for debugging and monitoring. **Code Example:** Example in C (using hiredis). This is fundamental for module development. """c #include <stdio.h> #include <stdlib.h> #include <string.h> #include <hiredis/hiredis.h> int main() { redisContext *c = redisConnect("localhost", 6379); if (c == NULL || c->err) { if (c) { printf("Connection error: %s\n", c->errstr); redisFree(c); } else { printf("Connection error: can't allocate redis context\n"); } exit(1); } redisReply *reply = redisCommand(c, "SET mykey myvalue"); if (reply == NULL) { printf("Error executing command: %s\n", c->errstr); redisFree(c); exit(1); } if (reply->type == REDIS_REPLY_ERROR) { printf("Redis error: %s\n", reply->str); } else { printf("SET command executed successfully\n"); } freeReplyObject(reply); redisFree(c); return 0; } """ ### 3.2. Memory Management **Do This:** Carefully manage memory allocation and deallocation. Avoid memory leaks and use appropriate data structures to minimize memory consumption. **Don't Do This:** Leak memory, leading to performance degradation and application instability. Use excessive memory without considering alternatives like Redis's optimized data structures. **Why:** Redis is an in-memory data store, and memory management is crucial for its performance and stability. Leaked memory can eventually lead to application crashes. Efficient memory usage maximizes the amount of data that can be stored in Redis. **Code Example:** String duplication with "strdup" and explicit "free". """c #include <stdlib.h> #include <string.h> char *duplicate_string(const char *str) { char *new_str = strdup(str); // Duplicate string, allocating memory if (new_str == NULL) { // Handle allocation failure (e.g., log error, return NULL) return NULL; } return new_str; } void free_string(char *str) { if (str != NULL) { free(str); // Deallocate memory } } //Usage: /* char *my_string = duplicate_string("Hello, Redis!"); if (my_string == NULL){ //Handle case of memory allocation failure } // ... use my_string ... free_string(my_string); //When done! */ """ ### 3.3. Logging **Do This:** Implement structured and informative logging to track application behavior and diagnose issues. **Don't Do This:** Use excessive or insufficient logging. Log sensitive information inappropriately. **Why:** Logs are essential for monitoring, debugging, and auditing. Structured logging (e.g., using JSON format) simplifies log analysis. **Code Example:** Basic logging with varying severity levels. """python import logging logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s') logging.debug('This is a debug message') # For detailed development information logging.info('This is an informational message') # Normal operations logging.warning('This is a warning message') # Potential issues logging.error('This is an error message') # Something went wrong logging.critical('This is a critical message') # Serious issue needing immediate attention try: result = 10 / 0 except Exception as e: logging.exception("An exception occurred") # Logs full stack trace """ ### 3.4. Concurrency and Atomicity **Do This:** Use Redis's atomic commands and transactions to ensure data consistency in concurrent environments. Employ optimistic locking with "WATCH" for complex operations. **Don't Do This:** Neglect concurrency issues, leading to race conditions and data corruption. Manually implement locking mechanisms when Redis's built-in features can provide atomic operations. **Why:** Redis is single-threaded, but concurrent clients can still cause race conditions if operations are not atomic. Transactions and "WATCH" provide mechanisms for ensuring data consistency in these scenarios. **Code Example:** Optimistic locking using "WATCH". This allows you to control changes during a multi-stage update. """python import redis r = redis.Redis(host='localhost', port=6379, db=0) def increment_counter(counter_key): while True: try: # Watch key for changes made by other clients between GET and SET with r.lock('increment_lock', blocking_timeout=5): #Example using lock (distributed) r.watch(counter_key) # Start watching the key current_value = r.get(counter_key) # Get current value if current_value is not None: new_value = int(current_value) + 1 else: new_value = 1 # Set to 1 if the key doesn't exist # Start a transaction and attempt to increment atomically pipe = r.pipeline() pipe.multi() # Start the transaction pipe.set(counter_key, new_value) # Set the new value result = pipe.execute() # Executes all commands in the pipeline atomically, returning the results if result: # result will be [True] if the transaction succeeded print(f"Counter incremented to {new_value}") return True except redis.WatchError: print("The key changed! Retrying...") # Key changed between GET and SET; retry the operation except redis.exceptions.LockError: print("Couldn't acquire lock - is another process updating the value?") return False #Or retry. finally: if r.connection: r.unwatch() # Stop watching (happens automatically with pipe.execute) """ ### 3.5. Use of Redis Modules **Do This:** When appropriate, leverage Redis Modules (e.g., RedisJSON, RedisSearch, RedisGraph) to extend Redis functionality and optimize performance. **Don't Do This:** Re-implement functionality readily available in existing Redis Modules. Use modules without properly understanding their impact on performance and stability. **Why:** Modules provide powerful extensions to Redis, enabling specialized data structures and query capabilities. They can significantly improve performance for specific use cases compared to implementing the same functionality in application code. **Example: Using RedisJSON** """python import redis import redis.commands.json.path as jsonpath r = redis.Redis(host='localhost', port=6379) r.json().set('user:123', jsonpath.ROOT_PATH, { 'name': 'John Doe', 'email': 'john.doe@example.com', 'age': 30 }) user_data = r.json().get('user:123', jsonpath.ROOT_PATH) print(user_data) # Output: {'name': 'John Doe', 'email': 'john.doe@example.com', 'age': 30} age = r.json().get('user:123', '.age') print(age) # Output 30 """ ### 3.6. Connection Management **Do This:** Use connection pooling and reuse connections to minimize connection overhead, especially with the latest client libraries. **Don't Do This:** Create a new Redis connection for every operation, which can be very expensive. **Why:** Establishing a new Redis connection has overhead. Connection reuse can significantly reduce the latency of Redis operations, particularly in frequently accessed parts of the application. Most modern Redis clients implement connection pooling. **Example:** Using connection pooling in Python (redis-py). """python import redis # Create a connection pool pool = redis.ConnectionPool(host='localhost', port=6379, db=0, max_connections=10) # Get a connection from the pool r = redis.Redis(connection_pool=pool) # Perform operations r.set('mykey', 'myvalue') value = r.get('mykey') print(value) #The connection will be returned to the pool when it goes out of scope. """ ## 4. Monitoring and Observability Effective monitoring and alerting are crucial for maintaining a healthy Redis deployment. ### 4.1. Key Performance Indicators (KPIs) **Do This:** Monitor key Redis metrics, such as memory usage, CPU utilization, connection count, and command latency. **Don't Do This:** Operate Redis without monitoring, making it difficult to identify and resolve performance bottlenecks or issues. **Why:** Monitoring KPIs allows you to proactively identify potential problems, optimize Redis configuration, and ensure the application's performance meets expectations. ### 4.2. Alerting **Do This:** Configure alerts based on critical metrics to notify you of potential issues, such as high memory usage or slow command latency. **Don't Do This:** Ignore critical alerts, leading to prolonged outages or performance degradation. **Why:** Alerting allows you to quickly respond to issues and prevent them from escalating into major problems. ### 4.3. Logging and Tracing **Do This:** Integrate distributed tracing to track requests across multiple services, including Redis. Use structured logging to easily query and analyze log data. **Don't Do This:** Rely solely on Redis logs for debugging, which can be difficult to correlate with application-level events. **Why:** Distributed tracing improves end-to-end visibility into application performance. Structured logging enables efficient log analysis and correlation. By adhering to these core architectural standards, Redis developers can build robust, performant, and maintainable applications that fully leverage the power of Redis. Continuous review and improvement of these standards are essential to adapting to new Redis features and evolving best practices.

# Deployment and DevOps Standards for Redis This document outlines the coding and operational standards for deploying and managing Redis in production environments. It is designed to guide developers and DevOps engineers in building robust, scalable, and secure Redis deployments. These standards are tailored to leverage the latest Redis features and best practices. ## 1. Build Processes and CI/CD ### 1.1 Build Automation **Standard:** Automate the Redis build process using tools like Make, CMake, or similar build systems. This ensures repeatable and consistent builds across different environments. **Do This:** Use Makefiles or CMakeLists.txt to define the build process. **Don't Do This:** Rely on manual build steps or environment-specific configurations without automation. **Why:** Automation reduces the risk of human error and ensures that builds are consistent regardless of the environment. **Example (Makefile):** """makefile REDIS_VERSION := 7.2.0 # Replace with the actual version all: redis-server redis-cli redis-server: wget http://download.redis.io/releases/redis-$(REDIS_VERSION).tar.gz tar xzf redis-$(REDIS_VERSION).tar.gz cd redis-$(REDIS_VERSION) && make redis-cli: @$(MAKE) -C redis-$(REDIS_VERSION) cli clean: rm -rf redis-$(REDIS_VERSION) redis-$(REDIS_VERSION).tar.gz install: cd redis-$(REDIS_VERSION) && sudo make install """ ### 1.2 Continuous Integration **Standard:** Integrate Redis builds into a Continuous Integration (CI) pipeline using tools like Jenkins, GitLab CI, GitHub Actions, or similar platforms. **Do This:** Define CI jobs that automatically build Redis from the source code, run unit tests, and perform basic integration tests. **Don't Do This:** Skip CI or rely on manual testing. **Why:** CI helps to identify issues early in the development cycle, reducing the cost and effort of fixing bugs later. **Example (GitHub Actions):** """yaml name: Redis CI on: push: branches: [ "main" ] pull_request: branches: [ "main" ] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Set up dependencies run: | sudo apt-get update sudo apt-get install -y gcc make tcl - name: Build Redis run: | make - name: Run Tests run: | make test """ ### 1.3 Unit and Integration Testing **Standard:** Maintain a comprehensive suite of unit and integration tests for Redis. These tests should cover core functionality, data types, and module integrations. **Do This:** Write unit tests using the "test" command available after building Redis. Add integration tests that simulate real-world usage scenarios. **Don't Do This:** Neglect testing or rely solely on manual testing. **Why:** Testing ensures that Redis functions correctly and that changes don't introduce regressions. **Example (Testing):** After 'make', running 'make test' executes the Redis test suite. Ensure all tests pass before deploying. ### 1.4 Code Review **Standard:** Implement a code review process where all code changes are reviewed by at least one other developer before being merged into the main branch. **Do This:** Use pull requests and code review tools to facilitate the review process. Focus on code quality, security, and adherence to the coding standards. **Don't Do This:** Merge code without review. **Why:** Code review improves code quality, identifies potential issues, and promotes knowledge sharing within the team. ### 1.5 Artifact Management **Standard:** Store built Redis binaries and configurations in an artifact repository such as Nexus, Artifactory, or cloud storage services like AWS S3 or Google Cloud Storage. **Do This:** Version your artifacts and use a consistent naming convention. Store build metadata and dependencies alongside the artifacts. **Don't Do This:** Store Redis binaries in a shared file system or rely on ad-hoc builds. **Why:** Artifact management provides a reliable and auditable way to manage Redis builds and deployments. ## 2. Deployment Strategies ### 2.1 Infrastructure as Code (IaC) **Standard:** Use Infrastructure as Code (IaC) tools such as Terraform, Ansible, or CloudFormation to define and manage Redis infrastructure. **Do This:** Define Redis server instances, network configurations, and security groups as code. Automate the provisioning and configuration of Redis clusters. **Don't Do This:** Manually provision and configure Redis infrastructure. **Why:** IaC enables repeatable and consistent deployments, reduces human error, and facilitates infrastructure management. **Example (Terraform):** """terraform resource "aws_instance" "redis_server" { ami = "ami-0c55b9633c061ef01" # Replace with a suitable AMI instance_type = "t3.medium" tags = { Name = "redis-server" } user_data = <<-EOF #!/bin/bash sudo apt-get update sudo apt-get install -y redis-server sudo systemctl enable redis-server EOF } """ ### 2.2 Configuration Management **Standard:** Use configuration management tools such as Ansible, Chef, or Puppet to automate the configuration of Redis instances. **Do This:** Define Redis configuration files as code and use configuration management to deploy and manage them. Automate tasks such as setting up replication, configuring persistence, and managing access controls. **Don't Do This:** Manually configure Redis instances. **Why:** Configuration management ensures that Redis instances are configured consistently and reduces the risk of configuration drift. **Example (Ansible):** """yaml --- - hosts: redis_servers tasks: - name: Install redis-server apt: name: redis-server state: present notify: Restart Redis handlers: - name: Restart Redis service: name: redis-server state: restarted """ ### 2.3 Deployment Automation **Standard:** Use deployment automation tools such as Jenkins, GitLab CI, or Argo CD to automate the deployment of Redis. **Do This:** Define deployment pipelines that build Redis from source code, run tests, and deploy the binaries and configurations to the target environment. **Don't Do This:** Manually deploy Redis. **Why:** Deployment automation reduces the risk of human error, ensures that deployments are repeatable, and facilitates faster release cycles. ### 2.4 Containerization **Standard:** Containerize Redis instances using Docker or similar containerization technologies. **Do This:** Create Docker images for Redis that include the Redis binaries, configuration files, and any required dependencies. Use Docker Compose or Kubernetes to orchestrate the deployment of Redis containers. **Don't Do This:** Deploy Redis directly on virtual machines or bare metal servers without containerization. **Why:** Containerization provides a consistent and isolated environment for Redis, simplifies deployment, and facilitates scalability. **Example (Dockerfile):** """dockerfile FROM ubuntu:latest RUN apt-get update && apt-get install -y redis-server COPY redis.conf /etc/redis/redis.conf EXPOSE 6379 CMD ["redis-server", "/etc/redis/redis.conf"] """ **Example (docker-compose.yml)** """yaml version: "3.9" services: redis: image: redis:latest ports: - "6379:6379" volumes: - redis_data:/data volumes: redis_data: """ ### 2.5 Orchestration (Kubernetes) **Standard:** Deploy Redis using Kubernetes for orchestration, scaling, and management. **Do This:** Use Helm charts or Kubernetes manifests to define Redis deployments. Leverage StatefulSets for managing stateful Redis instances. Use ConfigMaps and Secrets to manage configuration and credentials. Implement proper resource requests and limits. **Don't Do This:** Manually manage Redis instances in Kubernetes. **Why:** Kubernetes provides a scalable and resilient platform for running Redis clusters. **Example (Kubernetes Deployment):** """yaml apiVersion: apps/v1 kind: Deployment metadata: name: redis-deployment spec: replicas: 1 selector: matchLabels: app: redis template: metadata: labels: app: redis spec: containers: - name: redis image: redis:latest ports: - containerPort: 6379 """ ## 3. Production Considerations ### 3.1 Monitoring and Alerting **Standard:** Implement comprehensive monitoring and alerting for Redis using tools like Prometheus, Grafana, Datadog, or similar platforms. **Do This:** Monitor key Redis metrics such as memory usage, CPU utilization, network traffic, connection counts, and command latency. Set up alerts for abnormal conditions such as high memory usage, slow queries, or connection failures. Utilize Redis Insight or similar tools to visualize and analyze performance metrics. Also, leverage Redis slowlog to detect and optimize slow queries. The slowlog-log-slower-than config option defines the execution time, in microseconds, that a query must exceed to be logged. **Don't Do This:** Deploy Redis without monitoring and alerting. **Why:** Monitoring and alerting provide visibility into the health and performance of Redis, enabling proactive identification and resolution of issues. **Example (Prometheus Exporter):** Use the "redis_exporter" to expose Redis metrics to Prometheus. """bash docker run -d -p 9121:9121 oliver006/redis_exporter """ Configure Prometheus to scrape metrics from the exporter. Visualize metrics using Grafana dashboards. ### 3.2 High Availability **Standard:** Implement Redis Sentinel or Redis Cluster to ensure high availability and fault tolerance. **Do This:** Configure multiple Redis instances in a cluster or sentinel setup. Use automatic failover mechanisms to ensure that Redis remains available in the event of a failure. Consider using Redis Enterprise for managed high availability features. Configure Sentinel with the appropriate quorum and down-after-milliseconds to ensure proper failover behavior. **Don't Do This:** Rely on a single Redis instance for critical applications. **Why:** High availability ensures that Redis remains available even in the event of a failure, minimizing downtime and data loss. **Example (Redis Sentinel Configuration):** """ sentinel monitor mymaster 127.0.0.1 6379 2 sentinel down-after-milliseconds mymaster 5000 sentinel failover-timeout mymaster 10000 sentinel parallel-syncs mymaster 1 """ ### 3.3 Data Persistence **Standard:** Configure Redis persistence using RDB snapshots or AOF (Append Only File) to ensure data durability. Utilize Redis Enterprise Flash for tiered data storage. **Do This:** Choose the appropriate persistence strategy based on the application requirements. Configure RDB snapshots for point-in-time backups and AOF for incremental backups. Set up regular backups of the Redis data directory. Consider AOF rewrite to reduce the size of the append-only file. **Don't Do This:** Disable persistence for critical data requiring durability. **Why:** Data persistence ensures that Redis data is not lost in the event of a failure. **Example (redis.conf - AOF):** """ appendonly yes appendfilename "appendonly.aof" appendfsync everysec """ ### 3.4 Security **Standard:** Implement comprehensive security measures to protect Redis from unauthorized access and data breaches. **Do This:** * Enable Redis authentication using the "requirepass" option or ACLs (Access Control Lists) in newer Redis versions. Use strong, randomly generated passwords. * Bind Redis to specific network interfaces to limit network exposure. Use firewalls to restrict access to Redis ports. * Disable potentially dangerous commands such as "FLUSHALL" and "FLUSHDB" using the "rename-command" directive. * Keep Redis up to date with the latest security patches. * Use TLS encryption for client-server communication, especially in untrusted networks. * **Don't Do This:** * Expose Redis to the public internet without authentication or firewall protection. * Use default passwords or weak passwords. * Run Redis as the root user. * Implement the "CLIENT KILL" command to manage and terminate client connections that are idle or consuming excessive resources. **Why:** Security protects Redis data from unauthorized access and tampering. **Example (redis.conf - Authentication):** """ requirepass your_strong_password """ **Example (redis.conf - ACL):** """ acl setuser default on >your_strong_password allchannels allcommands loadmodule /path/to/redisgears.so # Example for modules """ ### 3.5 Resource Management **Standard:** Configure appropriate resource limits for Redis to prevent resource exhaustion. **Do This:** Set memory limits using the "maxmemory" option. Configure eviction policies to manage memory usage. Set CPU affinity and limit CPU usage. Use cgroups or similar mechanisms to limit the resources available to Redis containers. **Don't Do This:** Allow Redis to consume unlimited resources. **Why:** Resource management prevents Redis from consuming excessive resources, ensuring stability and performance. **Example (redis.conf - Memory Limit):** """ maxmemory 2gb maxmemory-policy allkeys-lru """ ### 3.6 Backup and Recovery **Standard:** Implement a regular backup and recovery plan for Redis. **Do This:** Schedule regular backups of the Redis data directory. Store backups in a secure and offsite location. Test the recovery process to ensure that it works. Consider using Redis Enterprise for managed backup and recovery features. Use "redis-cli --rdb" to backup and restore RDB files conveniently. Encrypt backups to further enhance data security at rest. Use redis-cli to automate the backup and restore processes. **Don't Do This:** Neglect backups or fail to test the recovery process. **Why:** Backup and recovery ensures that Redis data can be restored in the event of a failure or data loss. ### 3.7 Upgrades **Standard:** Plan and execute Redis upgrades in a controlled and phased manner. **Do This:** Test upgrades in a non-production environment before applying them to production. Use a rolling upgrade strategy to minimize downtime. Monitor the upgrade process closely and be prepared to roll back if necessary. Consult the Redis release notes for breaking changes and compatibility issues. **Don't Do This:** Apply upgrades without testing or planning. **Why:** Controlled upgrades minimize the risk of introducing issues and ensure that Redis remains available. ## 4. Monitoring and Observability Details ### 4.1 Key Metrics **Standard:** Continuously monitor the following key Redis metrics: * **Memory Usage:** Track "used_memory", "used_memory_rss", and "mem_fragmentation_ratio". High memory usage can lead to performance degradation and out-of-memory errors. Fragmentation can also impact performance. * **CPU Utilization:** Monitor CPU usage to identify CPU-bound workloads. Optimize queries or scale Redis instances to reduce CPU usage. * **Network Throughput:** Track network input and output to identify network bottlenecks. * **Connection Count:** Monitor the number of active client connections. High connection counts can indicate a connection leak or an overloaded Redis server. * **Command Latency:** Measure the latency of Redis commands to identify slow queries. Use the Redis slowlog to identify and optimize slow queries. * **Cache Hit Ratio:** Monitor the cache hit ratio to evaluate the effectiveness of Redis caching. ### 4.2 Alerting Thresholds **Standard:** Set up alerts for the following conditions: * **High Memory Usage:** Alert when memory usage exceeds a predefined threshold (e.g., 80% of "maxmemory"). * **High CPU Utilization:** Alert when CPU usage exceeds a predefined threshold (e.g., 80%). * **High Latency:** Alert when the latency of critical commands exceeds a predefined threshold (e.g., 100ms). * **Connection Errors:** Alert when the number of connection errors exceeds a predefined threshold. * **Replication Lag:** Alert when the replication lag between the primary and replica instances exceeds a predefined threshold. * **Eviction Rate:** Alert when there's a sustained increase in the rate of keys being evicted, as this might be a signal to re-evaluate the "maxmemory" setting or eviction policies. ### 4.3 Logging Standards **Standard:** Configure Redis to log important events and errors. **Do This:** * Set the "loglevel" configuration option to "notice" or "warning" to capture important events and errors. * Configure the "logfile" configuration option to specify the log file location. * Enable the slowlog to capture slow queries. Configure the "slowlog-log-slower-than" and "slowlog-max-len" configuration options to control the slowlog behavior. * Integrate Redis logs with a centralized logging system such as Elasticsearch, Splunk, or Graylog. **Don't Do This:** Disable logging or rely on manual log analysis. **Why:** Logging provides valuable information for troubleshooting and auditing. ## 5. Performance Optimization Tools & Techniques ### 5.1 Redis Insight **Standard:** Utilize Redis Insight, a free GUI, for visualizing and optimizing Redis data and performance. **Do This:** Install Redis Insight and connect it to your Redis instances. Use Redis Insight to monitor key metrics, analyze slow queries, and visualize data structures. **Why:** Redis Insight provides a user-friendly interface for monitoring and optimizing Redis. ### 5.2 redis-cli --hotkeys **Standard:** Periodically use the "redis-cli --hotkeys" command to identify the most frequently accessed keys. **Do This:** Run "redis-cli --hotkeys" to identify hotkeys. Optimize your application to reduce the load on hotkeys. **Why:** Identifying and optimizing hotkeys can significantly improve performance. ### 5.3 Redis Scan **Standard:** Use the SCAN command family (SCAN, HSCAN, SSCAN, ZSCAN) for iterating over keyspaces, hash fields, set members, and sorted set elements in a non-blocking manner. **Do This:** Implement "SCAN" to avoid blocking the server when iterating over large datasets or keyspaces. Use it in conjunction with commands like "DEL", "GET", "HGETALL", "SMEMBERS" etc. as needed. **Don't Do This:** Use "KEYS" command in production environments. **Why:** "SCAN" provides an efficient, non-blocking way to iterate through datasets, preventing performance degradation on large databases. **Example (SCAN):** """python import redis r = redis.Redis(host='localhost', port=6379, db=0) cursor = 0 while True: cursor, keys = r.scan(cursor=cursor, match='user:*', count=100) # Example: Match keys starting with 'user:' for key in keys: print(key) # Process the keys here if cursor == 0: break """ This well-structured document provides comprehensive standards for deploying and managing Redis effectively, incorporating best practices for build processes, CI/CD, deployment strategies, production considerations, monitoring, security, and performance optimization. The combination of clear instructions, actionable steps, and concrete code examples ensures that developers and DevOps engineers can confidently build, deploy, and maintain robust and secure Redis environments adhering to modern practices.

# Component Design Standards for Redis This document outlines the component design standards for Redis development. It aims to provide guidelines for creating reusable, maintainable, and efficient components within the Redis ecosystem, leveraging modern approaches and design patterns. ## 1. Introduction Effective component design is crucial for the long-term health and scalability of Redis-based applications and modules. Well-designed components promote code reuse, reduce complexity, improve maintainability, and enhance overall system performance. These standards are tailored specifically to the nuances of Redis, considering its single-threaded nature, data structures, and performance constraints. ## 2. High-Level Component Design Principles These principles establish a foundational approach to component design in Redis. * **Single Responsibility Principle (SRP):** A component should have one, and only one, reason to change. This enhances modularity and reduces the risk of unintended side effects. * **Do This:** Design components that encapsulate a specific, well-defined task or responsibility. For example, a component for managing rate limiting, or one for handling user sessions. * **Don't Do This:** Create "god classes" or components that handle multiple unrelated responsibilities. This leads to tightly coupled code that is difficult to modify or test. * **Open/Closed Principle (OCP):** A component should be open for extension, but closed for modification. This promotes adding new functionality without changing existing code, thus reducing the risk of introducing bugs. * **Do This:** Use interfaces, abstract classes, and configuration to allow extending the component's behavior without modifying its core code. Consider Redis Modules' event system for extensions. * **Don't Do This:** Directly modify existing components to add new features. This violates the OCP and can lead to instability. * **Liskov Substitution Principle (LSP):** Subtypes must be substitutable for their base types. This ensures that derived classes can be used anywhere their base classes are used without causing errors. * **Do This:** Ensure that derived classes adhere to the contract defined by their base classes (interfaces or abstract classes). Return the same types, or subtypes, as the methods they override or implement in the base class. * **Don't Do This:** Create derived classes with methods that throw exceptions when called in contexts where the base class's methods would function normally. This violates the LSP and can lead to unexpected runtime errors. * **Interface Segregation Principle (ISP):** Clients should not be forced to depend upon interfaces that they do not use. This promotes smaller, more cohesive interfaces and reduces dependencies. * **Do This:** Create multiple, specific interfaces instead of large, monolithic ones. A component interacting with a cache should only depend on the interface for caching operations, not all possible database operations. * **Don't Do This:** Define huge interfaces that contain many methods, some of which some client components might not use. * **Dependency Inversion Principle (DIP):** Depend upon abstractions, not concretions. High-level modules should not depend on low-level modules. Both should depend on abstractions. Abstractions should not depend upon details. Details should depend upon abstrations. * **Do This:** Use dependency injection or service location to provide components with the dependencies they need. Always depend on interfaces or abstract classes rather than concrete implementations. * **Don't Do This:** Directly instantiate concrete dependencies within a component. This creates tight coupling and makes it difficult to test or replace dependencies. ## 3. Component Types in Redis Projects Different types of components are used within Redis projects and require different considerations. * **Data Access Components:** These components are responsible for interacting with Redis to retrieve, store, and manipulate data. * **Business Logic Components:** These components implement the core business logic of the application, often operating on data retrieved from Redis. * **Utility Components:** This encompasses reusable functions or classes that address generic tasks, such as serialization, logging, or configuration management. * **Redis Modules:** Components built in C/C++ to extend Redis core functionality. * **Lua Scripts:** Scripts executed server-side to implement complex operations. * **Pub/Sub Handlers:** Components for handling messages in the Redis publish/subscribe system. * **Streams Consumers:** Processes that read and process data from Redis Streams. ## 4. Coding Standards for Data Access Components Data Access Components handle all interactions to and from the Redis database. They can use Redis clients to read data, write data, and invalidate data. * **Abstraction of Redis Interaction:** Separate the business logic from the direct interactions with Redis by using dedicated data access components (e.g., repositories or DAOs). """python # Example: Data Access Component in Python using redis-py import redis class UserRepository: def __init__(self, redis_client): self.redis_client = redis_client def get_user(self, user_id): user_data = self.redis_client.hgetall(f"user:{user_id}") if user_data: return {k.decode(): v.decode() for k, v in user_data.items()} # Decode bytes return None def save_user(self, user_id, user_data): self.redis_client.hmset(f"user:{user_id}", user_data) def delete_user(self, user_id): self.redis_client.delete(f"user:{user_id}") # Example Usage: redis_client = redis.Redis(host='localhost', port=6379, db=0) user_repository = UserRepository(redis_client) user = user_repository.get_user("123") if user: print(f"User data: {user}") else: print("User not found") new_user = {"name": "John Doe", "email": "john.doe@example.com"} user_repository.save_user("456", new_user) """ * **Why:** Simplifies testing because you only need to mock the data access layer, not the redis client and ensures the service layer does not contain any database-specific code. * **Error Handling:** Implement robust error handling to catch Redis connection errors, timeouts, or other exceptions. """python import redis class UserRepository: def __init__(self, redis_client): self.redis_client = redis_client def get_user(self, user_id): try: user_data = self.redis_client.hgetall(f"user:{user_id}") if user_data: return {k.decode(): v.decode() for k, v in user_data.items()} return None except redis.exceptions.ConnectionError as e: print(f"Error connecting to Redis: {e}") return None except redis.exceptions.TimeoutError as e: print(f"Redis timeout error: {e}") return None except Exception as e: print(f"An unexpected error occurred: {e}") return None """ * **Why:** Improves the robustness and reliability of the application. Without proper error handling, Redis connection problems can cause the application to crash. * **Connection Pooling:** Use connection pooling to efficiently manage Redis connections and reduce overhead. Most Redis clients (e.g., "redis-py") offer built-in connection pooling. """python import redis # Using Connection Pool redis_pool = redis.ConnectionPool(host='localhost', port=6379, db=0, max_connections=10) redis_client = redis.Redis(connection_pool=redis_pool) def get_data(key): try: return redis_client.get(key) except redis.exceptions.ConnectionError as e: print(f"Connection Error: {e}") return None data = get_data("mykey") """ * **Why:** Reduces the overhead of creating new connections for each operation by reusing pre-existing connections. This saves time if multiple processes are writing to the same Redis instance. * **Serialization/Deserialization:** Use a consistent and efficient serialization format (e.g., JSON, MessagePack) for storing complex data structures in Redis. When retrieving data, ensure correct deserialization. """python import redis import json class UserRepository: def __init__(self, redis_client): self.redis_client = redis_client def get_user(self, user_id): user_json = self.redis_client.get(f"user:{user_id}") if user_json: try: return json.loads(user_json.decode()) except json.JSONDecodeError as e: print(f"JSONDecodeError: {e}") return None return None def save_user(self, user_id, user_data): user_json = json.dumps(user_data) self.redis_client.set(f"user:{user_id}", user_json) """ * **Why:** Improves data consistency and allows storing structured data in Redis. JSON (or MessagePack) can represent complex data in a readable, standardized serialized format. * **Key Naming Conventions:** Establish and adhere to a consistent key naming convention to improve data organization and prevent key collisions. Use namespaces and delimiters (e.g., ""user:{user_id}:name""). """python # Consistent Key Naming USER_NAMESPACE = "user" def get_user_key(user_id): return f"{USER_NAMESPACE}:{user_id}:data" # Example Usage: user_key = get_user_key("123") # Output: "user:123:data" redis_client.get(user_key) """ * **Why:** Makes the data model more organized and prevents keys with similar names from colliding. Using functions to build keys centralizes data access and makes key refactoring easier. ## 5. Coding Standards for Business Logic Components These components contain the core logic of the application and should be separated as much as possible from infrastructure and framework details. * **Dependency Injection:** Use dependency injection to provide business logic components with their necessary dependencies (including data access components). """python # Example using a simple dependency injection class UserService: def __init__(self, user_repository): self.user_repository = user_repository def get_user_name(self, user_id): user = self.user_repository.get_user(user_id) if user: return user.get("name") return None # Usage: user_service = UserService(UserRepository(redis_client)) # Injecting UserRepository user_name = user_service.get_user_name("123") """ * **Why:** Improves testability and modularity by decoupling components such as services and data. Dependency injection increases code readability by making dependencies very clear. * **Transaction Management:** When dealing with multiple Redis operations that need to be atomic, use Redis transactions ("MULTI", "EXEC", "DISCARD") or Lua scripts. Lua scripts provide better performance for complex operations. """python import redis def transfer_funds(redis_client, from_account, to_account, amount): try: with redis_client.pipeline() as pipe: pipe.watch(from_account, to_account) # Optimistic locking from_balance = int(pipe.get(from_account) or 0) to_balance = int(pipe.get(to_account) or 0) if from_balance < amount: return False # Insufficient funds pipe.multi() pipe.set(from_account, from_balance - amount) pipe.set(to_account, to_balance + amount) pipe.execute() return True except redis.WatchError: # Key was modified, retry the transaction or handle failure print("WatchError: Key modified during transaction. Retrying...") return False except redis.exceptions.ConnectionError as e: print(f"Redis connection error: {e}") return False # Example Usage redis_client = redis.Redis(host='localhost', port=6379, db=0) success = transfer_funds(redis_client, "account1", "account2", 50) if success: print("Funds transferred successfully.") else: print("Funds transfer failed.") """ * **Why:** Ensures that multiple related operations are executed as a single atomic unit, preventing data corruption, and data inconsistencies in race scenarios. "WATCH" provides optimistic locking. * **Validation:** Validate input data before processing to prevent errors and ensure data integrity. Validation can occur via validation libraries. """python from cerberus import Validator schema = { 'name': {'type': 'string', 'required': True}, 'age': {'type': 'integer', 'min': 0}, 'email': {'type':

# Security Best Practices Standards for Redis This document outlines security best practices for Redis development to help developers create secure and maintainable Redis-based applications. It provides actionable guidelines, code examples, and explanations to minimize vulnerabilities and ensure data security. ## 1. Authentication and Authorization ### 1.1. Require Authentication **Standard:** Always enable authentication on your Redis instances. **Do This:** Enable "requirepass" in redis.conf. **Don't Do This:** Never deploy a Redis instance without authentication, especially in production environments. **Why:** Unauthenticated Redis instances are vulnerable to unauthorized access, allowing attackers to read, modify, or delete data, and potentially execute arbitrary commands on the server. **Code Example:** """redis # redis.conf requirepass your_strong_password """ **Anti-Pattern:** Using default passwords or weak passwords. **Modern Approaches:** Utilize strong, randomly generated passwords stored securely and rotated regularly. Consider integrating with external authentication providers for centralized management. ### 1.2. Use ACLs (Access Control Lists) **Standard:** Use ACLs (introduced in Redis 6) to grant granular permissions to users. **Do This:** Create users with specific permissions based on their required access. **Don't Do This:** Rely solely on "requirepass" for all users. Avoid granting excessive privileges. **Why:** ACLs provide finer-grained control over who can access what data and execute which commands, reducing the risk of privilege escalation. **Code Example:** """redis ACL SETUSER user1 +get +set ~object:* ACL SETUSER user2 +info +client """ **Anti-Pattern:** Granting "ALL COMMANDS" to all users. **Modern Approaches:** Leverage ACLs to implement the principle of least privilege. Regularly review and adjust permissions as needed. Use ACL categories (e.g., "@read", "@write", "@admin") to simplify permission management. ### 1.3 Restrict "RENAME" Command **Standard:** Carefully control access to the "RENAME" command, especially in shared environments. **Do This:** Deny "RENAME" access to users who don't need it. **Don't Do This:** Allow unrestricted use of "RENAME" on sensitive keys as this could lead to data corruption or information disclosure. **Why:** "RENAME" can overwrite existing keys, potentially causing data loss or security vulnerabilities. **Code Example:** """redis ACL SETUSER user3 -rename """ ### 1.4 Secure Redis Cluster **Standard:** Enable authentication and authorization across all nodes in a Redis Cluster. **Do This:** Configure "requirepass" and ACLs consistently on every node. Use the "CLUSTER" command appropriately, ensuring nodes are communicating securely. **Don't Do This:** Assume security on one node translates to security across the cluster. Ignoring security within cluster communication. **Why:** A compromised node in a cluster can compromise the entire dataset. Secure internal cluster communication prevents rogue nodes from joining and extracting information. ## 2. Network Security ### 2.1. Bind to Specific Interfaces **Standard:** Bind Redis to specific network interfaces, restricting access from outside the local network. **Do This:** Configure "bind" in redis.conf to listen only on specific IPs (e.g., "127.0.0.1" for local access, or a specific internal network IP). **Don't Do This:** Bind Redis to "0.0.0.0" (all interfaces) in production without proper firewall protection. **Why:** Binding to all interfaces exposes Redis to potential attacks from external networks. **Code Example:** """redis # redis.conf bind 127.0.0.1 10.0.0.10 """ **Anti-Pattern:** Binding to "0.0.0.0" without implementing a strong firewall. **Modern Approaches:** Employ a combination of interface binding, firewall rules, and network segmentation to restrict access. ### 2.2. Use Firewalls **Standard:** Implement firewall rules to allow only necessary traffic to Redis instances. **Do This:** Configure firewalls (e.g., iptables, ufw, cloud provider firewalls) to allow only trusted IP addresses and ports to communicate with Redis. **Don't Do This:** Rely solely on Redis authentication without a firewall. **Why:** Firewalls provide an additional layer of security, preventing unauthorized network access to Redis. **Code Example (iptables):** """bash iptables -A INPUT -s 10.0.0.0/24 -p tcp --dport 6379 -j ACCEPT iptables -A INPUT -p tcp --dport 6379 -j DROP """ **Modern Approaches:** Use cloud-native firewall solutions and network policies to manage network access dynamically. ### 2.3. Enable TLS Encryption **Standard:** Enable TLS (Transport Layer Security) encryption for client-server and cluster communication. This is supported natively in Redis 6 and later. **Do This:** Configure "tls-port", "tls-cert-file", "tls-key-file", and "tls-ca-cert-file" in redis.conf. **Don't Do This:** Use Redis over plain TCP in production environments, especially when transmitting sensitive data. **Why:** TLS encryption protects data in transit, preventing eavesdropping and man-in-the-middle attacks. **Code Example:** """redis # redis.conf tls-port 6380 tls-cert-file /path/to/redis.crt tls-key-file /path/to/redis.key tls-ca-cert-file /path/to/ca.crt """ **Modern Approaches:** Use automated certificate management tools (e.g., Let's Encrypt, HashiCorp Vault) to simplify certificate provisioning and rotation. Utilize mutual TLS (mTLS) for enhanced security, requiring clients to authenticate with certificates. ## 3. Input Validation and Sanitization ### 3.1. Sanitize User Input **Standard:** Sanitize any user-provided input before using it in Redis commands. **Do This:** Use appropriate escaping or encoding techniques to prevent command injection vulnerabilities. **Don't Do This:** Directly concatenate user input into Redis commands without sanitization. **Why:** Without proper sanitization, attackers can inject malicious commands into Redis. **Code Example (Python with redis-py):** """python import redis import redis.utils r = redis.Redis(host='localhost', port=6379, password='your_password') user_key = 'user:' + redis.utils.escape_string(user_id) # Sanitize user_id r.set(user_key, user_data) """ **Anti-Pattern:** Using Python's "str.format" or f-strings without escaping when constructing Redis commands. **Modern Approaches:** Leverage Redis client libraries that provide built-in sanitization and parameterization features. Consider using prepared statements where applicable (although Redis doesn't natively support them in the traditional SQL sense, client libraries often provide similar functionality). ### 3.2. Limit Command and Key Lengths **Standard:** Enforce reasonable limits on the length of keys and values to prevent denial-of-service attacks and memory exhaustion. **Do This:** Implement validation logic in your application to reject overly long keys or values. **Don't Do This:** Allow unbounded key lengths, which could lead to performance degradation or Redis crashes. **Why:** Extremely long keys or values can consume excessive memory and CPU resources. ## 4. Configuration and Operational Security ### 4.1. Disable Dangerous Commands **Standard:** Disable or rename potentially dangerous commands (e.g., "FLUSHALL", "FLUSHDB", "KEYS", "EVAL") in redis.conf. **Do This:** Use the "rename-command" directive in redis.conf to either disable or rename these commands. **Don't Do This:** Leave dangerous commands enabled in production environments. **Why:** These commands can be abused to delete data, expose sensitive information, or execute arbitrary Lua scripts. **Code Example:** """redis # redis.conf rename-command FLUSHALL "" rename-command FLUSHDB "" rename-command KEYS very_secret_key_enumeration_command rename-command EVAL "" """ **Modern Approaches:** Use ACLs in conjunction with command renaming for a more granular level of control. ### 4.2. Limit Memory Usage **Standard:** Configure "maxmemory" in redis.conf to limit the amount of memory Redis can use. Set an appropriate "maxmemory-policy" (e.g., "volatile-lru", "allkeys-lru") to evict keys when the memory limit is reached. **Do This:** Set "maxmemory" and choose a suitable "maxmemory-policy" based on your application's needs. **Don't Do This:** Allow Redis to consume unlimited memory, which can lead to out-of-memory errors and system instability. **Why:** Limiting memory usage prevents Redis from consuming all available system memory, ensuring stability and preventing denial-of-service conditions. **Code Example:** """redis # redis.conf maxmemory 2gb maxmemory-policy allkeys-lru """ ### 4.3. Secure Lua Scripting **Standard:** Carefully review and audit Lua scripts used with the "EVAL" command. Parameterize input to avoid injection vulnerabilities. Disable the "EVAL" command entirely if not needed. **Do This:** Validate and sanitize any user-provided input used within Lua scripts. Use "EVALSHA" to execute pre-compiled scripts. **Don't Do This:** Concatenate user input directly into Lua scripts. **Why:** Lua scripts can execute arbitrary code within the Redis server, so vulnerabilities in scripts can lead to severe security breaches. **Code Example:** """python import redis r = redis.Redis(host='localhost', port=6379, password='your_password') script = """ local key = KEYS[1] local value = ARGV[1] redis.call('SET', key, value) return redis.call('GET', key) """ sha = r.script_load(script) # Load the script result = r.evalsha(sha, 1, 'mykey', 'myvalue') # use EVALSHA for security print(result) """ **Modern Approaches:** Use server-side scripting with caution. Consider alternatives, such as Redis Modules, for more complex functionality with stricter security controls. ### 4.4 Regular Security Audits and Updates **Standard:** Regularly audit your Redis configuration, code, and dependencies for security vulnerabilities. Keep Redis updated to the latest stable version. **Do This:** Subscribe to security advisories and apply patches promptly. Perform regular security scans using vulnerability assessment tools. **Don't Do This:** Run outdated versions of Redis without applying security patches. **Why:** Regular audits and updates help identify and address security vulnerabilities before they can be exploited. ## 5. Data Encryption at Rest (Redis Enterprise and Modules) **Standard:** For sensitive data, consider using data encryption at rest. This is not natively supported in open-source Redis but is a standard feature of Redis Enterprise. Modules like RedisCrypt also provide encryption. **Do This:** If using Redis Enterprise, configure data encryption at rest. If using open-source Redis, evaluate and implement a suitable encryption module. **Don't Do This:** Store sensitive data in plain text without encryption. **Why:** Encryption at rest protects data from unauthorized access if the storage media is compromised. **Modern Approaches:** Use key management systems (KMS) to securely store and manage encryption keys. ## 6. Monitoring and Logging ### 6.1. Enable Comprehensive Logging **Standard:** Configure Redis to log all important events, including connection attempts, authentication failures, and command execution. **Do This:** Set the "loglevel" configuration option in redis.conf to "verbose" or "debug". Configure syslog integration for centralized logging. **Don't Do This:** Disable logging, or set the log level too low, which can hinder security investigations. **Why:** Comprehensive logging provides valuable information for security monitoring, incident response, and auditing. **Code Example:** """redis # redis.conf loglevel verbose """ ### 6.2. Monitor Redis Performance and Security **Standard:** Monitor Redis performance metrics (e.g., CPU usage, memory usage, connection count, slow queries) to detect anomalies that may indicate a security breach. Monitor security-related events (e.g., authentication failures, unauthorized command execution). **Do This:** Use tools like RedisInsight, Prometheus, Grafana, or cloud provider monitoring services to track Redis metrics. Set up alerts for suspicious activity. **Don't Do This:** Ignore Redis performance and security metrics. **Why:** Proactive monitoring helps detect and respond to security incidents in a timely manner. ## 7. Connection Pooling ### 7.1 Properly Configure Connection Pools **Standard:** Use and properly configure connection pools in your application to optimize resource utilization and prevent connection leaks. **Do This:** Ensure that the connection pool has appropriate limits for maximum and minimum connections, timeout settings, and connection health checks. **Don't Do This:** Create a new connection to Redis for every operation, as this can be resource-intensive and lead to performance bottlenecks. **Why:** Efficient connection pooling minimizes the overhead of establishing and tearing down connections, improving performance and stability. **Code Example (Python with redis-py):** """python import redis pool = redis.ConnectionPool(host='localhost', port=6379, db=0, password='your_password', max_connections=100) r = redis.Redis(connection_pool=pool) # Use the redis client 'r' as needed """ ### 7.2. Secure Connection Credentials **Standard:** Securely store and manage connection credentials, such as passwords, to prevent unauthorized access. **Do This:** Store passwords in environment variables, configuration files with restricted access, or secrets management systems (e.g., HashiCorp Vault). **Don't Do This:** Hardcode credentials directly in the application code. **Why:** Storing credentials securely prevents them from being exposed to unauthorized users. ## 8. Understand Redis Vulnerabilities ### 8.1 Stay Informed. **Standard:** Keep abreast of common Redis Vulnerabilities **Do This:** Monitor alerts from the Redis project, vulnerability databases (e.g. CVE), and cloud providers if hosting a managed instance. **Don't Do This:** Assume your current configuration is secure forever. New vulnerabilities are discovered constantly. **Why:** Knowing what to look for will improve code quality, configuration, and detection of malicious activity. ### 8.2 Protect Against Known Vulnerabilities **Standard:** Address known Redis vulnerabilities – especially around Lua scripting, command injection and default configurations. **Do This:** Follow recommendations from official Redis documentation on patching and mitigation. **Don't Do This:** Delay the application of security patches and recommended configuration changes, as this leaves systems vulnerable to exploitation. ## 9. Client-Side Security ### 9.1 Validate Data Retrieved from Cache **Standard:** Validate cache data on the client side **Do This:** Implement checks to ensure data is the expected type, within reasonable bounds, and matches with original schemas. **Don't Do This:** Blindly trust data pulled from the Redis cache. **Why:** Prevents vulnerabilities due to data corruption in the cache, particularly if external processes also have write access to Redis. ### 9.2 Secure Client Library Usage **Standard:** Use official, maintained client libraries and keep them up to date. **Do This:** Regularly update client libraries to benefit from latest bug fixes and performance improvements, and security patches. Pin library versions in deployment configurations. **Don't Do This:** Build your own client library. Use deprecated or unmaintained libraries. **Why:** Well vetted client libraries are less vulnerable and more performant than custom implementations.