Cline

# API Integration Standards for LLVM This document outlines the coding standards for integrating external APIs and backend services within the LLVM project. It focuses on patterns and practices that ensure maintainability, performance, and security, while adhering to the existing LLVM coding conventions. These guidelines aim to provide a consistent approach to API integration across the LLVM ecosystem. ## 1. Introduction Integrating LLVM components with external APIs and backend services requires careful consideration to maintain the project's stability, performance, and security. This document provides guidelines for creating robust, understandable, and maintainable integrations. It covers best practices for error handling, data serialization, asynchronous operations, authentication, and more. While LLVM traditionally avoids extensive external dependencies, certain tools and analyses may benefit significantly from external integration. This document addresses these scenarios, keeping the core LLVM principles in mind. ## 2. General Principles * **Minimize Dependencies:** Strive to minimize external dependencies. Evaluate the cost of introducing a new dependency against the benefits it provides. Consider if the functionality can be implemented within LLVM components. * **Do This:** Carefully evaluate whether an external dependency is truly necessary. * **Don't Do This:** Introduce dependencies without a thorough assessment of their impact on the project. * **Maintainability & Readability:** Code should be self-documenting, easy to understand, and follow LLVM's overall coding style. * **Do This:** Use meaningful variable and function names. Add comments explaining complex logic or integration points. Follow the LLVM coding style consistently. * **Don't Do This:** Write overly complex or cryptic code. Skimp on comments, especially around API calls. * **Error Handling:** Implement robust error handling to gracefully handle API failures. Log errors appropriately. * **Do This:** Use exception handling or checked error returns as appropriate. Provide informative error messages. * **Don't Do This:** Ignore error codes or exceptions. Allow exceptions to propagate uncaught. * **Security:** Protect against common security vulnerabilities (e.g., injection attacks, data breaches) when interacting with external APIs. * **Do This:** Validate all inputs from external APIs. Use secure communication protocols (HTTPS). Follow security best practices for the external platform. * **Don't Do This:** Trust data received from external APIs without validation. Store sensitive information unencrypted. ## 3. Connecting with Backend Services ### 3.1 Architectural Considerations * **Abstraction:** Introduce an abstraction layer to isolate the LLVM components from the specifics of the external API. This makes it easier to change the integration implementation or switch to a different API in the future. * **Why:** This reduces coupling and promotes a clear separation of concerns. * **Design Patterns:** Employ proven design patterns like Facade, Adapter, or Repository to structure the integration code. * **Why:** These patterns improve code organization, testability, and reduce code duplication. """c++ // Example: Facade pattern for API integration class ExternalAPI { public: virtual std::string fetchData(const std::string& query) = 0; virtual ~ExternalAPI() = default; }; class ConcreteExternalAPI : public ExternalAPI { public: std::string fetchData(const std::string& query) override { // Implementation to call the actual external API (e.g., using libcurl) // Example (placeholder): std::string result = "Data from external API for query: " + query; return result; } }; class LLVMDataService { public: LLVMDataService(ExternalAPI* api) : externalAPI(api) {} std::string retrieveData(const std::string& query) { // Perform LLVM-specific logic before calling the API llvm::outs() << "Preparing to fetch data for: " << query << "\n"; std::string data = externalAPI->fetchData(query); // Perform LLVM-specific logic after calling the API llvm::outs() << "Data retrieved successfully.\n"; return data; } private: ExternalAPI* externalAPI; }; // Usage: // ExternalAPI* apiImpl = new ConcreteExternalAPI(); // LLVMDataService service(apiImpl); // std::string data = service.retrieveData("some_query"); """ ### 3.2 Implementation Details * **HTTP Clients:** Use a well-established HTTP client library (e.g., "libcurl"). Ensure it is properly configured to handle TLS/SSL and other security concerns. Consider using a high-level library wrapper for easier use, but ensure it doesn't add significant overhead. * **Do This:** Choose a robust, widely used library with good support for security features. * **Don't Do This:** Roll your own HTTP client or use a deprecated library. * **Data Serialization:** Use a standardized data serialization format (e.g., JSON, Protocol Buffers). Use a library specifically designed for the format. * **Do This:** Choose a format appropriate for your data and performance requirements. Use "rapidjson" or "llvm::json" for smaller lightweight payloads where speed is critical. Use protobuf where schemas are well defined and versioning is a concern. * **Don't Do This:** Use custom or ad-hoc serialization formats. Serialize sensitive data without encryption. """c++ // Example: Using llvm::json for serializing and deserializing data #include "llvm/Support/JSON.h" #include <string> #include <vector> namespace llvm { void serializeData() { json::Object obj; obj["name"] = "Example Data"; obj["value"] = 42; obj["items"] = json::Array{1, 2, 3, 4, 5}; std::string jsonString = json::write(obj); llvm::outs() << "Serialized JSON: " << jsonString << "\n"; } void deserializeData(const std::string& jsonString) { Expected<json::Value> jsonValue = json::parse(jsonString); if (!jsonValue) { llvm::errs() << "Failed to parse JSON: " << toString(jsonValue.takeError()) << "\n"; return; } if (jsonValue->kind() != json::Value::Object) { llvm::errs() << "Expected JSON object.\n"; return; } json::Object& obj = *jsonValue->getAsObject(); std::string name = obj["name"]->getAsString().value(); int value = obj["value"]->getAsInteger().value(); json::Array& items = *obj["items"]->getAsArray(); llvm::outs() << "Name: " << name << "\n"; llvm::outs() << "Value: " << value << "\n"; llvm::outs() << "Items: "; for (auto& item : items) { llvm::outs() << item->getAsInteger().value() << " "; } llvm::outs() << "\n"; } } // namespace llvm // Usage: // llvm::serializeData(); // llvm::deserializeData(R"({"name":"Example Data","value":42,"items":[1,2,3,4,5]})"); """ * **Asynchronous Operations:** Use asynchronous operations to avoid blocking the main thread, especially for long-running API calls. Use "std::future" and "std::async". * **Do This:** Launch API calls in separate threads or use a non-blocking I/O model. * **Don't Do This:** Perform synchronous API calls on the main thread. * **Authentication:** Implement a secure authentication mechanism (e.g., OAuth 2.0, API keys). Store credentials securely (e.g., using a secrets manager). * **Do This:** Use established authentication protocols. Regularly rotate API keys. * **Don't Do This:** Hardcode credentials in the source code. Store credentials in plain text. Store secrets directly in git. ## 4. LLVM-Specific Considerations * **Integration Points:** Identify appropriate extension points within LLVM for integrating with external APIs. Common areas include: * **Passes:** Create a new pass to interact with the API. * **Analysis Utilities:** Extend analysis utilities to fetch data from external sources. * **Target-Specific CodeGen:** Modify target-specific code generation to leverage external services. * **LLVM Context:** Ensure the API calls do not interfere with the LLVM context or the overall compilation process. * **Do This:** Create a separate LLVM context for API-related operations (if necessary). Carefully synchronize access to shared resources. * **Don't Do This:** Directly modify the LLVM context from API callbacks without proper synchronization. * **Error Reporting:** Use LLVM's error reporting mechanisms to provide informative error messages related to API failures. * **Do This:** Use "llvm::errs()" and other LLVM error reporting tools to propagate errors to the user. * **Don't Do This:** Use "std::cerr" or other generic error streams. """c++ // Example: Reporting errors using llvm::errs() #include "llvm/Support/raw_ostream.h" void handleAPIError(const std::string& errorMessage) { llvm::errs() << "Error during API call: " << errorMessage << "\n"; } // Usage: // if (apiCallFailed) { // handleAPIError("Failed to retrieve data from the external service."); // } """ ## 5. Modern Approaches and Patterns * **gRPC:** Consider using gRPC for communication with backend services. gRPC is a high-performance, open-source universal RPC framework. Its advantages include: * **Protocol Buffers:** Uses Protocol Buffers for efficient serialization. * **Code Generation:** Automatically generates client and server code from protocol definitions. * **Multiple Languages:** Supports multiple programming languages. * **Microservices:** Design integrations following a microservices architecture. Decompose the API integration into smaller, independent services. * **Benefits:** Improved scalability, maintainability, and fault isolation. * **Considerations:** increased complexity and managing inter-service communication. * **Serverless Functions:** Use serverless functions (e.g., AWS Lambda, Azure Functions) to implement API integrations. * **Benefits:** Scalability, cost-effectiveness, and reduced operational overhead. ## 6. Common Anti-Patterns and Mistakes * **Tight Coupling:** Tightly coupling LLVM components with external APIs makes the code fragile and difficult to test/maintain. * **Ignoring Rate Limits:** Exceeding API rate limits can lead to service disruptions or being blocked. * **Lack of Monitoring:** Failing to monitor the health and performance of API integrations results in delayed problem detection and resolution. ## 7. Performance Optimization * **Caching:** Implement caching mechanisms to reduce the number of API calls, especially for frequently requested data. Use "llvm::StringMap" or other LLVM data structures for efficient storage. * **Do This:** Implement a cache with a reasonable expiration policy (TTL). Use a cache key that accurately reflects the data being cached. * **Don't Do This:** Cache sensitive data without proper encryption. Store an unbounded cache that grows indefinitely and consumes resources. * **Batching:** Batch multiple API requests into a single call to reduce network overhead. * **Why:** Reduces the overhead of multiple requests. * **Compression:** Enable data compression to reduce the size of data transmitted over the network. * **Why:** Reduces bandwidth usage and improves transfer speed. ## 8. Security Best Practices * **Input Validation:** Validate all inputs from external APIs to prevent injection attacks and other vulnerabilities. * **Do This:** Use whitelisting to allow only valid characters, formats, and lengths. Understand the specific validation requirements of the LLVM code consuming the external data. * **Don't Do This:** Trust data received from external APIs without validation. * **Data Encryption:** Encrypt sensitive data both in transit and at rest. Use TLS/SSL for communication over the network. * **Access Control:** Implement proper access control to restrict access to sensitive data and API endpoints. * **Regular Security Audits:** Conduct regular security audits to identify and address potential vulnerabilities. ## 9. Example: Integrating with a Simple REST API This example demonstrates a simplified integration with a REST API using "libcurl". It uses the Facade pattern to abstract the API interaction. """c++ #include "llvm/Support/raw_ostream.h" #include <curl/curl.h> #include <string> #include <stdexcept> namespace llvm { class RESTAPI { public: virtual std::string fetchData(const std::string& url) = 0; virtual ~RESTAPI() = default; }; class CurlRESTAPI : public RESTAPI { public: std::string fetchData(const std::string& url) override { std::string response; CURL* curl = curl_easy_init(); if (!curl) { throw std::runtime_error("Failed to initialize libcurl"); } curl_easy_setopt(curl, CURLOPT_URL, url.c_str()); curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writeCallback); curl_easy_setopt(curl, CURLOPT_WRITEDATA, &response); curl_easy_setopt(curl, CURLOPT_USERAGENT, "LLVM REST Client"); // Set a user agent CURLcode res = curl_easy_perform(curl); if (res != CURLE_OK) { curl_easy_cleanup(curl); throw std::runtime_error("curl_easy_perform() failed: " + std::string(curl_easy_strerror(res))); } curl_easy_cleanup(curl); return response; } private: static size_t writeCallback(void* contents, size_t size, size_t nmemb, std::string* output) { size_t totalSize = size * nmemb; output->append((char*)contents, totalSize); return totalSize; } }; class MyLLVMTool { public: MyLLVMTool(RESTAPI* api) : restAPI(api) {} std::string fetchDataFromAPI(const std::string& query) { try { std::string apiUrl = "https://api.example.com/data?q=" + query; std::string data = restAPI->fetchData(apiUrl); return data; } catch (const std::exception& e) { llvm::errs() << "Error fetching data: " << e.what() << "\n"; return ""; // Or handle the error more gracefully as needed } } private: RESTAPI* restAPI; }; } // namespace llvm // Usage: // llvm::RESTAPI* apiImpl = new llvm::CurlRESTAPI(); // llvm::MyLLVMTool tool{apiImpl}; // std::string data = tool.fetchDataFromAPI("some_query"); """ **Important Considerations:** * This example is simplified. Adapt it to your specific API requirements. * Replace ""https://api.example.com/data"" with the actual URL of the REST API. * Implement proper error handling (e.g., checking HTTP status codes). * Consider using a more robust JSON parsing library (e.g., "rapidjson", "llvm::json") for handling the API response. * Use dependency injection (as shown) to allow for easier testing with mock APIs. This document provides a comprehensive set of guidelines for integrating external APIs and backend services within LLVM. Adhering to these standards will help create robust, maintainable, and secure integrations that enhance the capabilities of the LLVM project.

# Component Design Standards for LLVM This document outlines the component design standards for LLVM, focusing on creating reusable, maintainable, and performant components within the LLVM ecosystem. It provides guidelines applicable to all LLVM subprojects, including the core compiler infrastructure, Clang, LLD, and related tools. These standards are designed to promote consistency, readability, and long-term maintainability of the LLVM codebase. ## 1. Principles of Component Design in LLVM ### 1.1 Abstraction and Encapsulation * **Do This:** Design components with clear abstractions that hide implementation details. * **Don't Do This:** Expose internal data structures or implementation details directly. **Why:** Abstraction simplifies the interface and reduces interdependence between components, making it easier to modify one component without affecting others. Encapsulation protects the internal state of a component from unintended external modification. **Code Example (Good):** """c++ // Good: Hiding implementation details behind an abstract interface. class TargetLoweringInfo { public: virtual ~TargetLoweringInfo() = default; virtual unsigned getRegForInlineAsm(const TargetRegisterClass *RC, MVT VT) const = 0; virtual CallingConv::ID getIRCallConv() const = 0; // ... other abstract methods }; // Concrete implementation, not exposed directly class AArch64TargetLowering : public TargetLoweringInfo { public: unsigned getRegForInlineAsm(const TargetRegisterClass *RC, MVT VT) const override { // ... implementation details for AArch64 return 0; // Dummy return } CallingConv::ID getIRCallConv() const override { //... return CallingConv::C; // dummy default } }; """ **Code Example (Bad):** """c++ // Bad: Exposing internal data structures. struct MyComponent { std::vector<int> internalData; // Directly accessible and modifiable. void processData(); // Public method. }; """ This example violates encapsulation by allowing direct access to the "internalData", which can lead to unintended side effects. ### 1.2 Single Responsibility Principle (SRP) * **Do This:** A component should have only one reason to change. * **Don't Do This:** Components should not be overly complex or perform unrelated tasks. Overly complex components are harder to understand, test, and maintain. **Why:** Following SRP makes components more focused, which improves their readability, testability, and reusability. **Code Example (Good):** """c++ // Good: Separate class for each responsibility. class InstructionSelector { public: MachineInstr *select(const BasicBlock &BB) { // ... instruction selection logic return nullptr; // Dummy return } }; class InstructionScheduler { public: void schedule(MachineInstr *instr) { // ... instruction scheduling logic } }; """ These classes are focused on their respective tasks: instruction selection and instruction scheduling. **Code Example (Bad):** """c++ // Bad: Combined responsibilities. class CompilerPass { public: void run(Module &M) { // ... instruction selection // ... instruction scheduling // ... register allocation } }; """ The "CompilerPass" class mixes instruction selection, scheduling, and register allocation, violating SRP. ### 1.3 Interface Segregation Principle (ISP) * **Do This:** Design interfaces that are specific to the clients that use them. * **Don't Do This:** Force clients to depend on interfaces they don't use. **Why:** ISP prevents unnecessary dependencies and reduces the impact of changes to interfaces. **Code Example (Good):** """c++ // Good: Segregated interfaces. class Printable { public: virtual void print() = 0; }; class Serializable { public: virtual void serialize() = 0; }; class MyClass : public Printable, public Serializable { public: void print() override { /*...*/ } void serialize() override { /*...*/ } }; """ Each interface focuses on a specific functionality. Components only implement the interfaces they need. **Code Example (Bad):** """c++ // Bad: Monolithic interface. class SomeInterface { public: virtual void methodA() = 0; virtual void methodB() = 0; virtual void methodC() = 0; }; class ClientA : public SomeInterface { public: void methodA() override { /*...*/ } void methodB() override { /*...*/ } void methodC() override { /* throw std::runtime_error("Not Implemented")*/ } // ClientA doesn't need this method, but is forced to implement it. }; """ This example forces "ClientA" to implement "methodC" even if it doesn't need it. ### 1.4 Dependency Inversion Principle (DIP) * **Do This:** Depend on abstractions, not concretions. * **Don't Do This:** Hardcode dependencies on concrete classes. **Why:** DIP promotes loose coupling and makes it easier to substitute implementations. **Code Example (Good):** """c++ // Good: Using dependency injection. class Logger { public: virtual void log(const std::string &message) = 0; }; class ConsoleLogger : public Logger { public: void log(const std::string &message) override { std::cout << message << std::endl; } }; class MyComponent { private: Logger *logger; public: MyComponent(Logger *logger) : logger(logger) {} void doSomething() { logger->log("Doing something..."); } }; """ The "MyComponent" depends on the "Logger" abstraction, not a concrete "ConsoleLogger". This allows substituting different loggers easily. **Code Example (Bad):** """c++ // Bad: Hardcoded dependency. class MyBadComponent { private: ConsoleLogger logger; // Hardcoded dependency on ConsoleLogger public: void doSomething() { logger.log("Doing something..."); } }; """ This example is tightly coupled to "ConsoleLogger", making it harder to test and reuse. ## 2. Component Structure and Organization ### 2.1 Directory Structure * **Do This:** Organize components into logical directories. * **Don't Do This:** Put all files in a single directory or create deeply nested directory structures that are hard to navigate. **Why:** A clear directory structure improves code discoverability and maintainability. LLVM generally follows a module-based structure. **Example:** """ llvm/ lib/ IR/ # LLVM Intermediate Representation AsmParser/ Core/ Instructions/ # Instruction definitions files (*.def) # Auto-generated instruction implementations (*.inc) Analysis/ # Static analyses Transforms/ # Transformations Scalar/ Vectorize/ include/ llvm/ IR/ Analysis/ Transforms/ tools/ llvm-as/ # LLVM assembler llvm-dis/ # LLVM disassembler """ ### 2.2 Naming Conventions * **Do This:** Use consistent naming conventions for files, classes, functions, and variables. * **Don't Do This:** Use cryptic or inconsistent names that are hard to understand. **Why:** Naming conventions improve code readability and reduce ambiguity. Names should be descriptive and follow LLVM's established conventions. **Examples:** * **Classes:** "ClassName" (CamelCase, starting with a capital letter) * **Functions:** "functionName" (camelCase, starting with a lowercase letter) * **Variables:** "variableName" (camelCase, starting with a lowercase letter) * **Files:** "ComponentName.cpp", "ComponentName.h" ### 2.3 Header Files * **Do This:** Use include guards to prevent multiple inclusions. Organize header files to minimize dependencies. Forward declare classes whenever possible. * **Don't Do This:** Include unnecessary header files. **Why:** Include guards prevent compilation errors. Minimizing includes reduces build times and dependencies. **Code Example:** """c++ // MyComponent.h #ifndef LLVM_MYCOMPONENT_H #define LLVM_MYCOMPONENT_H #include "llvm/Support/raw_ostream.h" // Only required headers namespace llvm { class MyClass; // Forward declaration class MyComponent { public: void doSomething(MyClass &obj); }; } // namespace llvm #endif // LLVM_MYCOMPONENT_H """ ## 3. Code Style and Formatting ### 3.1 LLVM Style * **Do This:** Follow the LLVM coding standards (see [LLVM Coding Standards](https://llvm.org/docs/CodingStandards.html)). Use "clang-format" to automatically format your code. * **Don't Do This:** Deviate from the LLVM coding standards. * **Why:** Consistency improves readability and reduces cognitive load. * Use 2-space indentation. * Keep lines reasonably short (around 80-120 characters). * Use descriptive comments. ### 3.2 Using "clang-format" * **Do This:** Configure "clang-format" with the LLVM style. Run it before committing code. * **Don't Do This:** Rely on manual formatting or ignore "clang-format" warnings. **Why:** "clang-format" automates code formatting and ensures compliance with LLVM's style guidelines. **Configuration:** LLVM uses a ".clang-format" file in the root of the project to define the style. **Usage:** """bash clang-format -i MyComponent.cpp # Format the file in-place """ ### 3.3 Comments and Documentation * **Do This:** Write clear and concise comments that explain the purpose and behavior of your code. Use Doxygen to generate API documentation. Provide usage examples. * **Don't Do This:** Write redundant comments that simply repeat what the code does. **Why:** Good comments improve code understanding and maintainability. Doxygen documentation makes it easy to generate API references automatically. **Code Example:** """c++ /** * @brief Calculates the average value of a vector of integers. * * @param values The vector of integers. * @return The average value, or 0.0 if the vector is empty. */ double calculateAverage(const std::vector<int> &values) { if (values.empty()) { return 0.0; } double sum = 0.0; for (int value : values) { sum += value; } return sum / values.size(); } """ ## 4. Error Handling and Assertions ### 4.1 LLVM Error Handling * **Do This:** Use "llvm::Error" to represent and propagate errors. Employ "Expected<T>" to represent a value that might be an error. * **Don't Do This:** Use exceptions for normal error handling. * **Why:** "llvm::Error" provides a consistent and efficient way to handle errors within LLVM. **Code Example:** """c++ #include "llvm/Support/Error.h" #include "llvm/Support/raw_ostream.h" llvm::Error doSomething(int value) { if (value < 0) { return llvm::make_error<llvm::StringError>("Value must be non-negative", llvm::inconvertibleErrorCode()); } // ... do something return llvm::Error::success(); } llvm::Expected<int> computeValue(int input) { if (input == 0) { return llvm::make_error<llvm::StringError>("Input cannot be zero", llvm::inconvertibleErrorCode()); } return input * 2; } int main() { if (auto Err = doSomething(-1)) { llvm::errs() << "Error: " << Err << "\n"; return 1; } auto Result = computeValue(10); if (!Result) { llvm::errs() << "Error: " << Result.takeError() << "\n"; return 1; } llvm::outs() << "Value: " << *Result << "\n"; return 0; } """ ### 4.2 Assertions * **Do This:** Use "llvm::support::llvm_unreachable" for cases that should never occur. Use "assert" liberally to check preconditions and invariants during development. * **Don't Do This:** Use assertions for handling expected errors or user input validation. **Why:** Assertions help catch bugs early in development. "llvm_unreachable" indicates that a code path is guaranteed to be unreachable. **Code Example:** """c++ #include "llvm/Support/raw_ostream.h" #include "llvm/Support/ErrorHandling.h" int getValue(int index) { assert(index >= 0 && index < 10 && "Index out of bounds"); // Precondition. if (index == 5) { llvm::support::llvm_unreachable("This should never happen!"); } return index * 2; } """ ## 5. Performance Considerations ### 5.1 Data Structures Selection * **Do This:** Choose appropriate data structures based on performance requirements. Consider using "llvm::SmallVector", "llvm::DenseMap", "llvm::SetVector", and other LLVM-specific data structures. * **Don't Do This:** Use standard library containers without considering performance implications. **Why:** LLVM-specific data structures are often optimized for common LLVM use cases. "SmallVector" avoids dynamic allocation for small numbers of elements. "DenseMap" is optimized for integer and pointer keys. **Code Example:** """c++ #include "llvm/ADT/SmallVector.h" void processValues(llvm::SmallVector<int, 32> &values) { // ... process values efficiently } #include "llvm/ADT/DenseMap.h" void processMap(llvm::DenseMap<int, int> &map) { // ... process map efficiently } """ ### 5.2 Memory Management * **Do This:** Use RAII (Resource Acquisition Is Initialization) to manage resources. Avoid manual memory management. When manual memory management is unavoidable, use smart pointers ("std::unique_ptr", "std::shared_ptr"). * **Don't Do This:** Use "new" and "delete" directly. Leak memory. **Why:** RAII and smart pointers automate resource management and prevent memory leaks. **Code Example:** """c++ #include <memory> class MyResource { public: MyResource() { /* Acquire resource */ } ~MyResource() { /* Release resource */ } }; void doSomething() { std::unique_ptr<MyResource> resource(new MyResource()); // Resource is automatically released when resource goes out of scope. } """ ### 5.3 Code Optimization * **Do This:** Profile your code. Use optimization flags ("-O2", "-O3"). Consider using profile-guided optimization (PGO). Minimize unnecessary computations. * **Don't Do This:** Optimize prematurely without profiling. Ignore performance bottlenecks. **Why:** Profiling identifies performance bottlenecks. Optimization flags and PGO improve code performance. ## 6. Testing ### 6.1 Unit Tests * **Do This:** Write unit tests for each component. Use LLVM's testing framework, including lit. * **Don't Do This:** Neglect testing. Commit code without tests. **Why:** Unit tests verify the correctness of components and prevent regressions. ### 6.2 Integration Tests * **Do This:** Write integration tests to ensure that components work together correctly. * **Don't Do This:** Assume that components will work together without testing. **Why:** Integration tests verify the interactions between components and catch integration issues. ### 6.3 Regression Tests * **Do This:** Add regression tests for each bug fix to prevent regressions. **Why:** Regression tests ensure that bug fixes are not inadvertently undone by future changes. ## 7. Concurrency and Thread Safety ### 7.1 Thread Safety * **Do This:** Design components to be thread-safe if they will be used in a multi-threaded environment. Use appropriate locking mechanisms ("llvm::sys::Mutex", "llvm::sys::LockGuard"). * **Don't Do This:** Share mutable state between threads without proper synchronization. Introduce race conditions. **Why:** Thread safety prevents data corruption and undefined behavior in concurrent environments. **Code Example:** """c++ #include "llvm/Support/Threading.h" class ThreadSafeComponent { private: llvm::sys::Mutex mutex; int state; public: int getState() { llvm::sys::LockGuard<llvm::sys::Mutex> lock(mutex); return state; } void setState(int newState) { llvm::sys::LockGuard<llvm::sys::Mutex> lock(mutex); state = newState; } }; """ Following these component design standards will result in a more robust, maintainable, and performant LLVM codebase. Adherence to these guidelines is crucial for fostering a healthy and collaborative development environment.

# Code Style and Conventions Standards for LLVM This document outlines the code style and conventions standards for the LLVM project. Adhering to these guidelines ensures code consistency, readability, and maintainability, which are crucial for a large and complex project like LLVM. These guidelines are intended to be used by both human developers and AI coding assistants to improve the quality and consistency of LLVM code. ## 1. General Formatting ### 1.1. Indentation and Whitespace * **Do This:** Use 2 spaces for indentation. Tabs should *never* be used. * **Don't Do This:** Use tabs or more than 2 spaces for indentation. * **Why:** Consistency in indentation is vital for readability. Two spaces provide a good balance between code nesting and horizontal space consumption. * **Example:** """c++ if (condition) { for (int i = 0; i < 10; ++i) { // Code within the loop doSomething(i); } } else { // Alternative code } """ ### 1.2. Line Length * **Do This:** Keep lines under 80 characters where practical. Aim for readability, and don't obsess over fitting everything into 80 characters if it harms clarity. * **Don't Do This:** Allow lines to routinely exceed 120 characters, making them hard to read on smaller screens or in diffs. * **Why:** Shorter lines improve readability and facilitate code review by allowing side-by-side comparisons in diff tools. * **Example:** """c++ // Good: Line split for readability Value *result = builder->CreateAdd(operand1, operand2, "sum"); // Bad: Long line, harder to read Value *result = builder->CreateAdd(operand1, operand2, "very_long_variable_name_that_makes_the_line_exceed_80_characters"); """ ### 1.3. Whitespace Usage * **Do This:** * Use a single space after keywords like "if", "for", "while", and "switch". * Use a single space around operators like "=", "+", "-", "*", "/", "==", "!=", "<", ">", "<=", ">=", "&&", "||". * Do not use spaces inside parentheses, brackets, or braces except where needed for clarity. * **Don't Do This:** * Omit spaces after keywords or around operators. * Add excessive spaces inside parentheses, brackets, or braces. * **Why:** Consistent whitespace improves readability and makes the code visually less cluttered. * **Example:** """c++ // Good if (x == 5) { y = z + 1; } // Bad if(x==5){ y=z+1; } """ ### 1.4. Vertical Whitespace * **Do This:** Use blank lines to separate logical blocks of code, such as function definitions, major sections within a function, and between different data structures. * **Don't Do This:** Overuse or underuse blank lines, resulting in either scattered or crammed code. * **Why:** Judicious use of vertical whitespace enhances the visual structure of the code, making it easier to understand. * **Example:** """c++ // Good void processData() { // Initialize variables int count = 0; std::vector<int> data; // Load data from file loadDataFromFile("data.txt", data); // Process data for (int value : data) { count += value; } // Print result std::cout << "Total: " << count << std::endl; } // Bad (crammed) void processData(){int count=0;std::vector<int> data;loadDataFromFile("data.txt",data);for(int value:data){count+=value;}std::cout<<"Total: "<<count<<std::endl;} """ ## 2. Naming Conventions ### 2.1. General Naming * **Do This:** * Use descriptive and meaningful names. * Prefer clear and explicit names over short and cryptic ones. * Be consistent in applying the same naming scheme across the project. * **Don't Do This:** * Use single-character variable names (except in very short loops). * Use abbreviations that are not widely understood in the LLVM community. * **Why:** Good naming significantly enhances code readability and reduces cognitive load. * **Example:**: "for (int i = 0; i < N; ++i)" is often fine, but "for (int elementIndex = 0; elementIndex < numberOfElements; ++elementIndex)" is easier to follow if the loop is more complex. ### 2.2. Variable Naming * **Do This:** Use "camelCase" for local variable names. * **Don't Do This:** Use "snake_case" or "PascalCase" for local variables. * **Why:** "camelCase" is a common convention in LLVM for local variables. * **Example:** """c++ int numberOfItems = 10; std::string itemName = "Example"; """ ### 2.3. Function Naming * **Do This:** Use "camelCase" for function names. Function names should generally be verbs or verb phrases indicating the action they perform. * **Don't Do This:** Use "snake_case" or "PascalCase" for function names. * **Why:** Consistency in function naming is important. Verb-based names accurately describe what functions do. * **Example:** """c++ int calculateSum(int a, int b) { return a + b; } void processData() { // ... } """ ### 2.4. Class and Struct Naming * **Do This:** Use "PascalCase" for class and struct names. * **Don't Do This:** Use "camelCase" or "snake_case" for class and struct names. * **Why:** "PascalCase" is the standard convention for class and struct names in LLVM. * **Example:** """c++ class MyClass { public: // ... }; struct DataStructure { int value; }; """ ### 2.5. Constant Naming * **Do This:** Use "PascalCase" for named constants (i.e. those defined with "static const"). Use all-uppercase "SCREAMING_SNAKE_CASE" for "#define" constants. Prefer "static const" over "#define" whenever possible. * **Don't Do This:** Use "camelCase" or "snake_case" for constants. * **Why:** Differentiating constants from variables helps in understanding the code. * **Example:** """c++ static const int MaxValue = 100; #define ARRAY_SIZE 256 """ ### 2.6. Enum Naming * **Do This:** Use "PascalCase" for enum names and "PascalCase" for enum values. * **Don't Do This:** Use "camelCase" or "snake_case" for enum names and values. * **Why:** Consistent enum naming enhances code clarity. * **Example:** """c++ enum class Color { Red, Green, Blue }; """ ### 2.7. Template Parameter Naming * **Do This:** Use a single uppercase letter, or a descriptive name starting with an uppercase letter. When a descriptive name is used, it should match the concept that the template parameter represents. * **Don't Do This:** Use unclear abbreviations. * **Why:** Template parameters should be easily identifiable within the template. * **Example:** """c++ template <typename T> T add(T a, T b) { return a + b; } template <typename ElementType> class MyVector { // ... }; """ ## 3. Comments ### 3.1. General Commenting * **Do This:** * Write clear and concise comments to explain complex logic and design decisions. * Keep comments up-to-date with code changes. * Use proper grammar and spelling in comments. * **Don't Do This:** * Write obvious comments that simply restate the code. * Leave outdated or incorrect comments. * Use excessive jargon or abbreviations without explanation. * **Why:** Comments are essential for understanding the code's purpose and functionality. Clear and accurate comments reduce maintenance effort and improve collaboration. ### 3.2. Doxygen-Style Comments * **Do This:** Use Doxygen-style comments for documenting functions, classes, and files. * **Don't Do This:** Neglect to document the purpose, parameters, and return values of functions and classes. * **Why:** Doxygen-style comments allow automatic documentation generation, which is crucial for large projects like LLVM. * **Example:** """c++ /** * @brief Calculates the sum of two integers. * * This function adds two integers and returns the result. * * @param a The first integer. * @param b The second integer. * @return The sum of a and b. */ int calculateSum(int a, int b) { return a + b; } """ ### 3.3. Inline Comments * **Do This:** Use inline comments to explain specific lines or blocks of code that are not immediately obvious. * **Don't Do This:** Overuse inline comments for trivial code. * **Why:** Inline comments clarify complex logic or non-obvious operations. * **Example:** """c++ for (int i = 0; i < 10; ++i) { // Multiply i by 2 to get the even number int evenNumber = i * 2; // ... } """ ## 4. Code Structure and Design ### 4.1. Function Length * **Do This:** Keep functions reasonably short (typically under 50 lines). If a function becomes too long, refactor it into smaller, more manageable functions. * **Don't Do This:** Write very long functions that perform multiple unrelated tasks. * **Why:** Shorter functions are easier to understand, test, and maintain. They also promote code reuse. ### 4.2. Class Design * **Do This:** * Adhere to the Single Responsibility Principle (SRP): each class should have one specific responsibility. * Use proper encapsulation: keep internal state private and provide access through public methods. * Use inheritance and polymorphism appropriately to model relationships between classes. * **Don't Do This:** * Create "god classes" that do everything. * Expose internal state directly. * Overuse inheritance, leading to complex and fragile class hierarchies. * **Why:** Good class design promotes modularity, reusability, and maintainability. ### 4.3. Error Handling * **Do This:** * Use exceptions for exceptional cases (e.g., invalid input, resource allocation failure). * Handle errors gracefully and provide informative error messages. * Use "llvm::Error" to handle recoverable errors. * **Don't Do This:** * Ignore errors or handle them silently. * Use exceptions for normal control flow. * **Why:** Robust error handling is crucial for preventing crashes and providing a good user experience. * **Example:** """c++ #include "llvm/Support/Error.h" #include "llvm/Support/raw_ostream.h" llvm::Error processData(int value) { if (value < 0) { return llvm::make_error<llvm::StringError>("Invalid value: " + std::to_string(value), llvm::inconvertibleErrorCode()); } // Process data llvm::outs() << "Processing value: " << value << "\n"; return llvm::Error::success(); } int main() { llvm::Error err1 = processData(10); if (err1) { llvm::errs() << "Error: " << llvm::toString(std::move(err1)) << "\n"; } llvm::Error err2 = processData(-5); if (err2) { llvm::errs() << "Error: " << llvm::toString(std::move(err2)) << "\n"; } return 0; } """ ### 4.4. Resource Management * **Do This:** * Use RAII (Resource Acquisition Is Initialization) to manage resources like memory, file handles, and locks. * Use smart pointers ("std::unique_ptr", "std::shared_ptr") to automatically manage dynamically allocated memory. * **Don't Do This:** * Manually allocate and deallocate memory using "new" and "delete" without proper RAII. * Leak resources. * **Why:** RAII ensures that resources are properly released, even in the presence of exceptions, preventing resource leaks and improving code reliability. ### 4.5. Modern C++ Features * **Do This:** * Use modern C++ features like lambda expressions, range-based for loops, and auto type deduction. * Prefer "constexpr" for compile-time constants. * Use move semantics to avoid unnecessary copying of objects. * **Don't Do This:** * Rely on deprecated C++ features. * Write verbose code that can be simplified with modern features. * **Why:** Modern C++ features can significantly improve code readability, efficiency, and safety. * **Example:** """c++ #include <iostream> #include <vector> int main() { std::vector<int> data = {1, 2, 3, 4, 5}; // Range-based for loop for (int value : data) { std::cout << value << " "; } std::cout << std::endl; // Lambda expression auto multiplyByTwo = [](int x) { return x * 2; }; std::vector<int> multipliedData; for (int value : data) { multipliedData.push_back(multiplyByTwo(value)); } for (int value : multipliedData) { std::cout << value << " "; } std::cout << std::endl; // Auto type deduction auto sum = 0; for (auto value : data) { sum += value; } std::cout << "Sum: " << sum << std::endl; return 0; } """ ## 5. LLVM-Specific Guidelines ### 5.1. LLVM Coding Conventions * **Do This:** * Follow the existing coding style and conventions of the specific LLVM component you are working on. * Look at existing files or recent commits within the specific directory to determine local conventions. * **Don't Do This:** * Introduce new coding styles that are inconsistent with the rest of the component. * **Why:** Consistency within a component is crucial for maintainability and readability. ### 5.2. LLVM Data Structures * **Do This:** * Use LLVM-specific data structures like "SmallVector", "StringRef", and "ArrayRef" where appropriate. These are optimized for common LLVM use cases. * **Don't Do This:** * Use standard library containers ("std::vector", "std::string") without carefully considering performance implications. * **Why:** LLVM data structures are designed to be efficient and memory-friendly for specific LLVM tasks. ### 5.3. LLVM Diagnostic Infrastructure * **Do This:** * Use the LLVM diagnostic infrastructure ("llvm::DiagnosticInfo", "llvm::SourceMgr", "llvm::LLVMContext") for reporting errors, warnings, and remarks. * **Don't Do This:** * Print diagnostic messages directly to "std::cerr" or "std::cout". * **Why:** The LLVM diagnostic infrastructure provides a consistent and extensible way to report diagnostic information, allowing tools to handle diagnostics in a uniform manner. ### 5.4. LLVM Pass Infrastructure * **Do This:** * Use the LLVM pass infrastructure for implementing compiler passes. * Follow the standard pass structure, including the "runOnFunction" or "runOnModule" methods. * **Don't Do This:** * Implement custom pass management mechanisms. * **Why:** The LLVM pass infrastructure provides a uniform and efficient way to implement and manage compiler passes. ### 5.5. Including LLVM Headers * **Do This:** Include headers using angle brackets "<>" for system headers and LLVM headers that are part of the LLVM distribution. Use quotes """" for headers that are local to your project or component. Order includes alphabetically within each category. * **Don't Do This:** Mix include styles or use relative paths for LLVM distribution headers. * **Why:** This helps distinguish standard library headers from project-specific headers and keeps include paths clean and maintainable. **Example:** """c++ #include <algorithm> #include <iostream> #include <vector> #include "llvm/ADT/SmallVector.h" #include "llvm/IR/Function.h" #include "llvm/Pass.h" #include "MyComponent/MyHeader.h" """ ## 6. Security Best Practices ### 6.1. Input Validation * **Do This:** Validate all external inputs to prevent vulnerabilities such as buffer overflows, format string bugs, and code injection. * **Don't Do This:** Assume that external inputs are safe or well-formed. * **Why:** Input validation is essential for preventing security vulnerabilities. ### 6.2. Memory Safety * **Do This:** * Use memory-safe programming techniques (e.g., bounds checking, smart pointers). * Be careful when using raw pointers and manual memory management. * **Don't Do This:** * Write code that is prone to buffer overflows, use-after-free errors, or other memory-related vulnerabilities. * **Why:** Memory safety is crucial for preventing security exploits. ### 6.3. Integer Overflows * **Do This:** Check for integer overflows when performing arithmetic operations, especially when dealing with sizes and indices. * **Don't Do This:** Assume that integer arithmetic is always safe. * **Why:** Integer overflows can lead to unexpected behavior and security vulnerabilities. ### 6.4. Safe String Handling * **Do This:** Use safe string handling functions (e.g., "llvm::StringRef::startswith", "llvm::StringRef::endswith") to avoid buffer overflows and format string bugs. * **Don't Do This:** Use unsafe string functions like "strcpy" or "sprintf". * **Why:** Safe string handling prevents common security vulnerabilities related to string manipulation. ## 7. Performance Optimization ### 7.1. Data Locality * **Do This:** Design data structures and algorithms to maximize data locality, which can improve cache utilization and reduce memory access latency. * **Don't Do This:** Access memory in a random or scattered manner. * **Why:** Data locality can significantly improve performance, especially for memory-bound applications. ### 7.2. Avoiding Unnecessary Copies * **Do This:** Use move semantics to avoid unnecessary copying of objects. * **Don't Do This:** Pass large objects by value when they can be passed by reference or moved. * **Why:** Copying large objects can be expensive, especially when they are not modified. ### 7.3. Efficient Algorithms * **Do This:** Choose efficient algorithms (e.g., sorting, searching) that are appropriate for the specific task. * **Don't Do This:** Use naive or inefficient algorithms that can lead to poor performance. * **Why:** Algorithm choice has a significant impact on performance. ### 7.4. Profiling * **Do This:** Use profiling tools to identify performance bottlenecks and optimize the most critical sections of code. Consider using LLVM's built-in profiling capabilities. * **Don't Do This:** Guess at performance issues without empirical evidence. * **Why:** Profiling provides valuable insights into performance bottlenecks and helps focus optimization efforts on the most critical areas. ## 8. Tooling and Automation ### 8.1. Clang-Format * **Do This:** Use "clang-format" to automatically format code according to the LLVM coding style. Configure your IDE or editor to automatically run clang-format on save. * **Don't Do This:** Manually format code or ignore "clang-format" warnings. * **Why:** "clang-format" ensures consistent code formatting and reduces the burden of manual formatting. ### 8.2. Clang-Tidy * **Do This:** Use "clang-tidy" to automatically check code for style violations, potential bugs, and security vulnerabilities. Configure your build system to run "clang-tidy" as part of the build process. * **Don't Do This:** Ignore "clang-tidy" warnings or disable checks without a valid reason. * **Why:** "clang-tidy" helps identify and fix issues early in the development cycle. ### 8.3. Continuous Integration * **Do This:** Use a continuous integration (CI) system to automatically build, test, and analyze code changes before they are merged into the main branch. * **Don't Do This:** Merge code changes without proper CI testing. * **Why:** CI ensures that code changes do not break the build, introduce new bugs, or violate coding standards. By adhering to these code style and conventions standards, LLVM developers can contribute to a more consistent, readable, and maintainable codebase, ultimately leading to a better and more secure compiler infrastructure. These standards are intended to guide both human developers and AI coding assistants in producing high-quality LLVM code.

# Deployment and DevOps Standards for LLVM This document outlines deployment and DevOps standards for LLVM projects. It focuses on build processes, continuous integration/continuous deployment (CI/CD), and production considerations. The goal is to ensure LLVM projects are built, tested, and deployed efficiently, reliably, and securely. ## 1. Build Processes A well-defined build process is crucial for producing consistent and reproducible artifacts. LLVM uses CMake as its primary build system. ### 1.1. CMake Standards CMake is fundamental to building LLVM. Proper usage ensures portability and maintainability. **Do This:** * Use CMake targets extensively. This makes dependencies explicit and simplifies the build graph. * Employ generator expressions ("$<...>") for conditional compilation based on build configurations (Debug, Release, etc.). * Use CMake modules to encapsulate common build logic. * Leverage features provided by the "LLVM-Config.cmake" module (installed with LLVM) within projects using LLVM. * Prefer "target_link_libraries", "target_include_directories", "target_compile_definitions", and "target_compile_features" over global settings. **Don't Do This:** * Avoid direct manipulation of compiler flags (e.g., setting "CXX_FLAGS" directly). Use CMake's built-in mechanisms instead. * Don't overuse "execute_process" for tasks that can be handled by CMake commands. * Don't hardcode paths; rely on CMake variables (e.g., "CMAKE_SOURCE_DIR", "CMAKE_BINARY_DIR"). **Why:** CMake provides a platform-independent and well-structured way to manage the build process. Adhering to these standards ensures portability and simplifies maintainability. **Example:** """cmake # CMakeLists.txt cmake_minimum_required(VERSION 3.13) # Ensure a modern CMake version project(MyProject) # Find LLVM find_package(LLVM REQUIRED CONFIG) message(STATUS "Found LLVM ${LLVM_PACKAGE_VERSION}") include_directories(${LLVM_INCLUDE_DIRS}) add_executable(MyTool MyTool.cpp) # Link against LLVM libraries target_link_libraries(MyTool LLVMSupport LLVMCore) # Add compile definitions based on build type: target_compile_definitions(MyTool PRIVATE $

# Testing Methodologies Standards for LLVM This document outlines the testing methodologies standards for LLVM development, focusing on unit, integration, and end-to-end testing. Adhering to these standards will enhance code quality, maintainability, and reliability, and is designed for use by developers and AI coding assistants. ## 1. Introduction A robust testing strategy is crucial for LLVM's stability and correctness. This document provides guidelines for designing, implementing, and executing tests at different levels, from individual units to the entire toolchain. The goal is to ensure that new code integrates seamlessly, existing functionality remains intact, and the overall system performs optimally. ## 2. Unit Testing Unit tests focus on individual components or functions in isolation. They verify that a specific piece of code behaves as expected under various conditions. ### 2.1 Standards * **Do This:** Write unit tests for all non-trivial functions and classes. * **Why:** Unit tests provide fast feedback on the correctness of code changes, preventing defects from propagating to higher levels. * **Don't Do This:** Skip unit tests for "simple" code. Edge cases and subtle bugs often hide in seemingly straightforward functions. * **Why:** Even simple code can have bugs, and unit tests provide documentation of the expected behavior. * **Do This:** Use a testing framework like Google Test (gtest) that is already well integrated in LLVM. * **Why:** Consistent frameworks improve code readability, maintainability and intergration. ### 2.2 Implementation * **Directory Structure:** Unit tests should reside in a "unittests" subdirectory next to the source code they test. For example, if the source file is "lib/Transforms/Scalar/MyPass.cpp", the unit tests should be in "lib/Transforms/Scalar/unittests/MyPass/MyPassTest.cpp". * **Naming Conventions:** Test files should follow the "[ClassName]Test.cpp" pattern. Test classes should be named "[ClassName]Test". Each test case should describe a specific scenario. ### 2.3 Example: Unit Testing a Simple Pass Consider a simple LLVM pass that converts all "add" instructions to "sub" instructions (for demonstration purposes only). **Source Code ("lib/Transforms/Scalar/Add2Sub.cpp"):** """c++ #include "llvm/Pass.h" #include "llvm/IR/Function.h" #include "llvm/IR/Instructions.h" #include "llvm/Transforms/Scalar.h" #include "llvm/Support/raw_ostream.h" #include "llvm/IR/InstIterator.h" using namespace llvm; namespace { struct Add2Sub : public FunctionPass { static char ID; Add2Sub() : FunctionPass(ID) {} bool runOnFunction(Function &F) override { bool modified = false; for (auto &BB : F) { for (auto &I : BB) { if (auto *Add = dyn_cast<BinaryOperator>(&I)) { if (Add->getOpcode() == Instruction::Add) { Value *Op1 = Add->getOperand(0); Value *Op2 = Add->getOperand(1); BinaryOperator *Sub = BinaryOperator::CreateSub(Op1, Op2, "addtosub", Add); Add->replaceAllUsesWith(Sub); Sub->insertBefore(Add); Add->eraseFromParent(); modified = true; } } } } return modified; } }; } char Add2Sub::ID = 0; static RegisterPass<Add2Sub> X("add2sub", "Add to Sub Pass", false, false); FunctionPass* llvm::createAdd2SubPass() { return new Add2Sub(); } """ **Unit Test ("lib/Transforms/Scalar/unittests/Add2Sub/Add2SubTest.cpp"):** """c++ #include "llvm/Transforms/Scalar/Add2Sub.h" // Include the header file for your pass #include "llvm/AsmParser/Parser.h" // Include the parser library #include "llvm/IR/Module.h" #include "llvm/Support/SourceMgr.h" #include "gtest/gtest.h" // Google Test framework using namespace llvm; namespace { std::unique_ptr<Module> parseIR(const char *IR) { LLVMContext Context; SMDiagnostic Err; std::unique_ptr<Module> Mod = parseAssemblyString(IR, Err, Context); if (!Mod) { Err.print("Add2SubTest", errs()); } return Mod; } TEST(Add2SubTest, BasicTest) { const char* IR = "define i32 @test_func(i32 %a, i32 %b) {\n" " %1 = add i32 %a, %b\n" " ret i32 %1\n" "}\n"; std::unique_ptr<Module> Mod = parseIR(IR); ASSERT_TRUE(Mod != nullptr); Function *F = Mod->getFunction("test_func"); ASSERT_TRUE(F != nullptr); Add2Sub pass; bool modified = pass.runOnFunction(*F); ASSERT_TRUE(modified); std::string str; raw_string_ostream OS(str); Mod->print(OS, nullptr); // Expected output after the add instruction is replaced with sub const char* ExpectedIR = "; ModuleID = '<string>'\n" "source_filename = \"<string>\"\n" "\n" "define i32 @test_func(i32 %a, i32 %b) {\n" " %addtosub = sub i32 %a, %b\n" " ret i32 %addtosub\n" "}\n"; ASSERT_STREQ(OS.str().c_str(), ExpectedIR); } } """ **CMakeLists.txt (in "lib/Transforms/Scalar/unittests/Add2Sub/")**: """cmake add_llvm_unittest(Add2SubTest Add2SubTest.cpp) """ ### 2.4 Common Anti-Patterns * **Testing implementation details:** Unit tests should focus on the public interface and observable behavior, not the internal implementation. * **Why:** Testing implementation details makes tests brittle and prone to failure when the implementation changes. * **Ignoring edge cases:** Ensure tests cover boundary conditions, null inputs, and other exceptional scenarios. * **Why:** Edge cases are often sources of bugs that can lead to crashes or incorrect results. * **Not using assertions correctly:** Use GTest assertions (ASSERT_TRUE, ASSERT_EQ, etc.) correctly to clearly indicate the expected outcome. * **Why:** Incorrect assertions can lead to misleading test results. ### 2.5 Modern Best Practices * **Fuzzing:** Use LLVM's integrated fuzzing tools to automatically generate test cases and uncover unexpected behavior. * **Why:** Fuzzing can find bugs that are difficult to identify through manual test creation. * **Property-based testing:** Define properties that should hold true for a range of inputs and automatically generate tests to verify them. * **Why:** Property-based testing can improve test coverage and find subtle bugs. ## 3. Integration Testing Integration tests verify that different components of the LLVM toolchain work together correctly. This includes testing interactions between passes, the compiler front-end, and the code generator. ### 3.1 Standards * **Do This:** Write integration tests to verify that changes in one component do not break functionality in other components. * **Why:** Integration tests catch subtle bugs that can arise from interactions between different parts of the system. * **Don't Do This:** Rely solely on unit tests. Integration tests are essential for verifying end-to-end behavior. * **Why:** Unit tests only verify individual components, not their interactions. ### 3.2 Implementation * **Test Cases:** Integration tests typically involve compiling a source file, running it through a series of LLVM passes, and verifying the output. * **FileCheck:** Use "FileCheck" for verifying the output of the compiler or other tools. "FileCheck" allows you to specify patterns that should be present in the output. * **lit:** Use "lit" (LLVM Integrated Tester) to manage and run integration tests. "lit" provides a consistent way to execute tests and report results. ### 3.3 Example: Integration Testing a Pass with Clang Consider an integration test that verifies that the "Add2Sub" pass correctly transforms a C source file. **C Source File ("test.c"):** """c int add(int a, int b) { return a + b; } """ **LLVM IR File ("test.ll"):** *Generated by clang -S -emit-llvm test.c* """llvm ; ModuleID = 'test.c' source_filename = "test.c" target datalayout = "e-m:e-p270:32:32-p271:32:32-i64:64-i128:128-f80:128-fp128:128-n64:64-S128" target triple = "x86_64-unknown-linux-gnu" ; Function Attrs: noinline nounwind optnone uwtable define i32 @add(i32 %0, i32 %1) #0 { %3 = alloca i32, align 4 %4 = alloca i32, align 4 store i32 %0, i32* %3, align 4 store i32 %1, i32* %4, align 4 %5 = load i32, i32* %3, align 4 %6 = load i32, i32* %4, align 4 %7 = add nsw i32 %5, %6 ret i32 %7 } attributes #0 = { noinline nounwind optnone uwtable "frame-pointer"="all" "min-legal-vector-width"="0" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+cmov,+cx16,+inline-plt,+mmx,+sse,+sse2,+x87" "tune-cpu"="generic" } !llvm.module.flags = !{!0, !1, !2} !llvm.ident = !{!3} !0 = !{i32 2, !"SDK Version", [8 x i8] c"12.3 (sdk)\00"} !1 = !{i32 2, !"clang version", !4} !2 = !{i32 1, !"wchar_size", i32 4} !3 = !{!"clang version 15.0.7"} !4 = !{!"15.0.7"} """ **Test File ("test.verify"):** """llvm ; RUN: clang -S -emit-llvm %s -o - | opt -load %{BUILD_DIR}/lib/LLVMAdd2Sub.so -add2sub | FileCheck %s ; CHECK: %addtosub = sub nsw i32 %5, %6 define i32 @add(i32 %0, i32 %1) { %3 = alloca i32, align 4 %4 = alloca i32, align 4 store i32 %0, i32* %3, align 4 store i32 %1, i32* %4, align 4 %5 = load i32, i32* %3, align 4 %6 = load i32, i32* %4, align 4 %7 = add nsw i32 %5, %6 ret i32 %7 } """ **Explanation:** * "RUN:" line specifies the command to run the test. It pipes LLVM IR generated by clang to "opt", which loads the Add2Sub dynamic library, runs the add2sub pass on it, and pipes the output to "FileCheck". * "CHECK:" line specifies the pattern that should be present in the output of the command. **CMakeLists.txt (in "lib/Transforms/Scalar/"):** """cmake add_llvm_library(LLVMAdd2Sub MODULE Add2Sub.cpp) llvm_add_integration_test(Add2SubTest test.verify DEPENDS LLVMAdd2Sub ) """ ### 3.4 Common Anti-Patterns * **Overlapping test coverage:** Ensure that integration tests cover areas not already covered by unit tests. * **Why:** Redundant tests waste resources and can make it harder to isolate the root cause of a failure. * **Complex test setups:** Keep integration tests as simple as possible to reduce the risk of test failures due to unrelated issues. * **Why:** Complex setups can make it difficult to diagnose test failures. * **Ignoring compiler warnings:** Address all compiler warnings in test code to ensure that the tests are valid and reliable. * **Why:** Compiler warnings can indicate potential bugs or undefined behavior in the test code. ### 3.5 Modern Best Practices * **Continuous Integration:** Integrate integration tests into a continuous integration system to automatically run tests on every code change. * **Why:** Continuous integration provides early feedback on integration issues and prevents regressions. * **Docker:** Use Docker containers to create a consistent test environment and ensure that tests are reproducible. * **Why:** Docker containers eliminate environment-specific issues and make it easier to run tests on different platforms. ## 4. End-to-End Testing End-to-end tests verify the entire LLVM toolchain, from source code input to executable output. These tests ensure that the compiler, assembler, linker, and runtime environment work together correctly. ### 4.1 Standards * **Do This:** Write end-to-end tests for complex scenarios and critical features. * **Why:** End-to-end tests provide the highest level of confidence in the correctness of the LLVM toolchain. * **Don't Do This:** Rely solely on end-to-end tests. They are slow and can be difficult to debug. * **Why:** Unit and integration tests provide faster and more targeted feedback. ### 4.2 Implementation * **Test Cases:** End-to-end tests typically involve compiling a large source file or set of files, linking them together, and running the resulting executable. * **Regression Tests:** Use regression tests to ensure that known bugs do not reappear in future versions of LLVM. * **Performance Tests:** Use performance tests to measure the impact of code changes on the performance of the LLVM toolchain. ### 4.3 Example: End-to-End Testing with a Complex Program Consider an end-to-end test that compiles and runs a complex program, such as a benchmark or a large application. **Source Code ("complex.c"):** """c #include <stdio.h> int main() { int sum = 0; for (int i = 0; i < 1000000; ++i) { sum += i; } printf("Sum: %d\n", sum); return 0; } """ **Test File ("complex.test"):** """llvm ; REQUIRES: clang ; RUN: %clang %s -o %t ; RUN: %t | FileCheck %s ; CHECK: Sum: 499999500000 """ **Explanation:** * "REQUIRES: clang" indicates that the test requires Clang. * "RUN: %clang %s -o %t" compiles the C source file using Clang and creates an executable named "%t". "%s" is a placeholder for the path to the test file, and "%t" is a temporary file name. * "RUN: %t | FileCheck %s" runs the executable and pipes the output to "FileCheck". * "CHECK: Sum: 499999500000" verifies that the output of the executable contains the expected sum. **CMakeLists.txt :** """cmake llvm_add_end_to_end_test(ComplexTest complex.test complex.c) """ ### 4.4 Common Anti-Patterns * **Unclear test objectives:** Ensure that each end-to-end test has a clear and well-defined objective. * **Why:** Vague test objectives can make it difficult to determine whether a test has passed or failed. * **Lack of test data:** Provide sufficient test data to exercise all relevant code paths. * **Why:** Insufficient test data can lead to incomplete test coverage and missed bugs. ### 4.5 Modern Best Practices * **Test-Driven Development (TDD):** Write tests before writing the code. * **Why:** Guides the writing process and ensures that the code being developed is testable.. * **Mutation Testing:** Systematically introduce small defects (mutations) into the code and verify that the tests can detect them. * **Why::** Mutation testing can help improve the quality and effectiveness of tests. ## 5. Performance Testing Performance tests measure the speed, memory usage, and other performance characteristics of the LLVM toolchain. These tests are essential for identifying and addressing performance regressions. ### 5.1 Standards * **Do This:** Write performance tests for critical code paths and frequently used tools. * **Why:** Performance tests provide early warning of performance regressions and help optimize the LLVM toolchain. * **Don't Do This:** Ignore performance regressions. Investigate and address them promptly. * **Why:** Performance regressions can have a significant impact on the overall performance of the LLVM toolchain. ### 5.2 Implementation * **Benchmarking Tools:** Use dedicated benchmarking tools, such as Google's benchmark library, to measure performance accurately. * **Statistical Analysis:** Use statistical analysis techniques to identify statistically significant performance differences. * **Regression Testing:** Use performance regression testing to automatically detect performance regressions in new code changes. * **perf:** Use "perf" to profile code execution and determine performance bottlenecks. ### 5.3 Example: Performance Testing with Google Benchmark """c++ #include <benchmark/benchmark.h> static void BM_StringCreation(benchmark::State& state) { for (auto _ : state) { std::string empty_string; } } // Register the function as a benchmark BENCHMARK(BM_StringCreation); // Define another benchmark static void BM_StringCopy(benchmark::State& state) { std::string x = "hello"; for (auto _ : state) { std::string copy(x); } } BENCHMARK(BM_StringCopy); // Run the benchmark BENCHMARK_MAIN(); """ To use this, it needs to be built and linked with the google benchmark library. ### 5.4 Security Testing Security testing is crucial in LLVM, particularly due to its role in code compilation and optimization. This testing ensures LLVM is robust against vulnerabilities. * **Do This:** Employ fuzzing techniques and static analysis. * **Why:** Early detection of vulnerabilities prevents exploitation in compiled code. * **Don't Do This:** Neglect input validation and sanitization. * **Why:** Improper handling of inputs can lead to security breaches. * **Do This:** Conduct memory safety checks consistently, particularly where user-provided data affects memory operations. * **Why:** Prevents common vulnerabilities like buffer overflows. ## 6. Conclusion By adhering to these testing methodologies standards, LLVM developers can ensure the quality, reliability, and security of the LLVM toolchain. A comprehensive testing strategy, encompassing unit, integration, end-to-end, and performance tests, is essential for maintaining the stability and performance of LLVM. This ensures that changes are robust, reliable, and do not introduce regressions. The integration of modern best practices, such as fuzzing, continuous integration, and test-driven development, will keep the LLVM project at the forefront of compiler technology.