Cline

# Tooling and Ecosystem Standards for Crystal This document outlines the recommended tooling and ecosystem standards for Crystal development. These guidelines promote consistency, maintainability, and performance across projects, and serve as context for AI coding assistants. ## 1. Dependency Management with Shards Shards is the official dependency manager for Crystal. Using it correctly is crucial for project maintainability. ### 1.1. Using Shards.lock **Do This:** Always commit "shards.lock" to your version control system. **Why:** "shards.lock" ensures that everyone on the team uses the exact same versions of dependencies. Without it, you risk dependency conflicts and inconsistent behavior across environments. **Example:** """ # Correct - shards.lock is present .gitignore shards.yml shards.lock src/ """ **Don't Do This:** Exclude "shards.lock" from your Git repository. ### 1.2. Specifying Dependency Versions **Do This:** Use semantic versioning when specifying dependencies in "shards.yml". Utilize version constraints like "~>", ">=", and "<" to define acceptable version ranges. **Why:** Semantic versioning allows you to receive bug fixes and minor features without introducing breaking changes. Specific version constraints provide control over which versions are allowed, mitigating risks associated with unexpected updates. **Example:** """yaml # shards.yml dependencies: kemal: github: kemalcr/kemal version: "~> 1.5" # Allow versions 1.5.x up to (but not including) 1.6.0 jwt: github: mosop/jwt version: ">= 2.0.0" # Allow versions 2.0.0 and greater """ **Don't Do This:** Use vague version specifiers like "*" or no version specifier at all. This can lead to unpredictable dependency resolution. Avoid overly strict version pinning unless absolutely necessary due to compatibility issues. ### 1.3. Keeping Dependencies Up-to-Date **Do This:** Regularly update your dependencies using "shards update". Review the changelogs of updated dependencies for potential breaking changes and adjust your code accordingly. **Why:** Keeping dependencies up-to-date ensures that you benefit from bug fixes, performance improvements, and security patches. Reviewing changelogs helps you proactively address any compatibility issues introduced by updates. **Example:** """bash # Update all dependencies shards update # Update a specific shard shards update kemal """ **Don't Do This:** Neglect dependency updates for extended periods. This increases the risk of security vulnerabilities and compatibility issues. Also, blindly run "shards update" without reviewing changelogs. ### 1.4 Private Shards and Repositories **Do This:** When working with private dependencies, use appropriate authorization mechanisms (e.g., SSH keys, access tokens) and ensure that your deployment environment has the necessary permissions to access the private repositories. Store credentials safely using environment variables or dedicated secrets management tools. **Why:** Secure access management is critical for protecting sensitive code held in private repositories. Following a least-privilege principle minimizes the risk of unauthorized access to confidential dependencies. **Example:** """yaml # shards.yml repository: type: git url: git@github.com:my-org/private-shard.git branch: main private: true """ **Don't Do This:** Hardcode credentials in your "shards.yml", environment variables, or source code. Grant excessive permissions for private shard access to users or environments. ## 2. Code Formatting and Linting Consistent code formatting and linting are essential for code readability and maintainability. ### 2.1. Using "crystal tool format" **Do This:** Use "crystal tool format" to automatically format your code according to the standard Crystal style. Integrate this command into your development workflow (e.g., as a pre-commit hook). **Why:** "crystal tool format" enforces a consistent code style, reducing subjective formatting debates and making the code easier to read and understand. **Example:** """bash crystal tool format src/**/*.cr """ **Don't Do This:** Rely on manual formatting only. This is time-consuming and prone to inconsistencies. ### 2.2. Using "ameba" for Linting **Do This:** Use "ameba" (a Crystal linter) to identify potential code quality issues, style violations and potential errors. Integrate "ameba" into your CI/CD pipeline to automatically check code quality on every commit or pull request. **Why:** "ameba" helps you catch common mistakes and enforce coding standards, leading to cleaner, more maintainable code. **Example:** 1. Add "ameba" as a dependency in "shards.yml": """yaml dependencies: ameba: github: veelenga/ameba version: "~> 1.0" """ 2. Run "shards install". 3. Run "ameba" to analyze your code: """bash ameba """ 4. Configure ".ameba.yml" to customize linting rules according to project conventions. **Don't Do This:** Ignore linting warnings or disable "ameba" checks. ### 2.3. Editor Integration **Do This:** Configure your code editor (e.g., VS Code, Atom, Sublime Text) to automatically format your code on save and display linting errors. Use plugins such as the Crystal VS Code extension. **Why:** Editor integration makes code formatting and linting seamless and automatic, improving developer productivity and code quality. **Example:** * **VS Code:** Install the "Crystal Language" extension and configure it to run "crystal tool format" on save. Configure "ameba" integration for real-time linting feedback. **Don't Do This:** Rely solely on command-line tools for formatting and linting. ## 3. Testing Frameworks and Strategies Comprehensive testing is crucial for ensuring the reliability of your Crystal code. ### 3.1. Using the Built-in "spec" Framework **Do This:** Use the built-in "spec" framework for writing unit tests and integration tests. **Why:** "spec" provides a simple and expressive way to define tests in Crystal. **Example:** """crystal # spec/my_class_spec.cr require "spec" require "../src/my_class" describe MyClass do it "should return 42" do my_object = MyClass.new my_object.answer.should eq 42 end end """ **Don't Do This:** Neglect writing tests or use ad-hoc testing methods. ### 3.2. Test-Driven Development (TDD) **Do This:** Consider adopting a TDD approach, writing tests before writing the actual code. **Why:** TDD helps you clarify requirements and design better APIs. It also ensures that your code is testable from the start. **Example:** 1. Write a failing test for a feature """crystal # spec/calculator_spec.cr require "spec" require "../src/calculator" describe Calculator do it "should add two numbers correctly" do calc = Calculator.new calc.add(2, 3).should eq 5 end end """ 2. Implement only the necessary code to pass the above test. """crystal # src/calculator.cr class Calculator def add(a : Int32, b : Int32) : Int32 a + b end end """ 3. Refactor **Don't Do This:** Write code without tests, or write tests only after the code is complete. ### 3.3. Using Mocks and Stubs **Do This:** Use mocks and stubs to isolate units of code during testing. Libraries like "test_double" can assist. **Why:** Mocks and stubs allow you to test code in isolation, without relying on external dependencies or databases. **Example:** """crystal # spec/my_service_spec.cr require "spec" require "../src/my_service" require "test_double" describe MyService do it "should call the external API" do api_mock = mock(ExternalApi) expect(api_mock).to receive(:get_data).and_return("mocked data") service = MyService.new(api_mock.object) result = service.process_data result.should eq "processed mocked data" end end """ **Don't Do This:** Rely heavily on real external dependencies in your tests. ### 3.4. Integration Testing **Do This:** Write integration tests to verify that different parts of your application work correctly together. Test interactions with databases, external APIs, and other systems. **Why:** Integration tests catch issues that unit tests might miss, such as incorrect data mappings or network connectivity problems. **Example:** Write integration tests to see if kemal web application correctly interacts with database. This might require accessing the real database. **Don't Do This:** Only perform unit testing and ignore integration testing entirely. ## 4. Logging and Monitoring Proper logging and monitoring are essential for debugging and identifying issues in production. ### 4.1. Using the Standard Library "log" **Do This:** Use the standard library "log" module for logging. Configure different log levels (e.g., "DEBUG", "INFO", "WARN", "ERROR") to control the verbosity of your logs. **Why:** The "log" module provides a simple and standard way to log messages in Crystal. **Example:** """crystal require "log" Log.info "Application started" Log.debug "Processing request: #{request.inspect}" Log.warn "High CPU usage detected" """ **Don't Do This:** Use "puts" or "print" for logging in production code. These methods are not configurable and can clutter the output. ### 4.2. Structured Logging **Do This:** Use structured logging formats (e.g., JSON) to make logs easier to parse and analyze. Consider using a library like "logstash" with appropriate configurations. **Why:** Structured logs enable you to easily filter, aggregate, and analyze logs using tools like Elasticsearch, Graylog, or Splunk. **Example:** """crystal require "json" require "log" data = { "event": "user_login", "user_id": 123, "timestamp": Time.utc } Log.info(data.to_json) """ **Don't Do This:** Log unstructured text only. This makes it difficult to automate log analysis. ### 4.3. Centralized Logging **Do This:** Configure your application to send logs to a centralized logging system (e.g., ELK stack, Graylog, Papertrail). **Why:** Centralized logging makes it easier to monitor and analyze logs from multiple instances of your application. **Example:** Use Logstash in combination with Elasticsearch and Kibana to collect, process and visualize logs from multiple Crystal app instances. **Don't Do This:** Rely solely on local log files. This makes it difficult to troubleshoot issues in distributed environments. ### 4.4. Monitoring Tools **Do This:** Integrate your application with monitoring tools (e.g., Prometheus, Grafana, Datadog) to track key performance metrics (e.g., CPU usage, memory usage, response time). **Why:** Monitoring tools provide real-time insights into the health and performance of your application. **Example:** Use Prometheus client libraries to expose application metrics and Grafana for dashboard visualization. **Don't Do This:** Operate applications in production without adequate monitoring. ## 5. Concurrency and Parallelism Crystal provides powerful concurrency features. Using them correctly is important for performance. ### 5.1. Using Fibers **Do This:** Use fibers for concurrent operations that are I/O-bound (e.g., network requests, database queries). **Why:** Fibers are lightweight and efficient, allowing you to handle many concurrent operations without creating excessive overhead. Crystal's scheduler will correctly manage fiber execution. **Example:** """crystal require "http/client" def fetch_url(url) Fiber.new do response = HTTP::Client.get(url) puts "Fetched #{url}: #{response.status_code}" end.resume end urls = ["https://crystal-lang.org", "https://kemalcr.com", "https://www.google.com"] urls.each { |url| fetch_url(url) } sleep 1 # Wait for fibers to complete """ **Don't Do This:** Use threads for I/O-bound operations. Threads are heavier than fibers and may not scale as well. ### 5.2. Using Channels **Do This:** Use channels for communicating between fibers safely and efficiently. **Why:** Channels provide a way to send data between fibers without risking race conditions or data corruption. **Example:** """crystal channel = Channel(String).new Fiber.new do channel.send("Hello from Fiber 1") end.resume Fiber.new do message = channel.receive puts "Fiber 2 received: #{message}" end.resume sleep 1 # Wait for fibers to complete """ **Don't Do This:** Share mutable data between fibers without proper synchronization. ### 5.3. Parallel Processing **Do This:** Use parallel processing for CPU-bound operations to take advantage of multiple cores. **Why:** Parallel processing can significantly improve performance for computationally intensive tasks. Crystal supports parallel processing with the "spawn" keyword. **Example:** """crystal def process_data(data : Array(Int32)) : Array(Int32) data.map { |x| x * 2 } end data = (1..1000).to_a # Naive implementation result = process_data(data) # Parallelized implementation result_parallel = Parallel.map(data) { |x| x * 2 } """ **Don't Do This:** Over-parallelize operations. This can lead to increased overhead and decreased performance. Measure and profile to find the optimal level of parallelism. Avoid unnecessary data copying during parallel operations. ## 6. Tooling for Specific Frameworks ### 6.1 Kemal When devoloping web application in Crystal, Kemal is often choice. **Do This:** Use Kemal CLI tools for scaffolding new projects and generating code. Understand Kemal's middleware architecture and use it to structure request processing logic. **Example:** """bash kemal new my_app cd my_app kemal serve """ **Don't Do This:** Reinvent the wheel by not using Kemal's built-in features for common tasks. ### 6.2 ORM (Object-Relational Mapping) **Do This:** Use an ORM like Granite or Crinja for database interactions instead of writing raw SQL queries. **Why:** ORMs abstract away database-specific details, improve code readability, and reduce the risk of SQL injection vulnerabilities. **Example (Granite):** """crystal require "granite" require "pg" Granite::Base.connection_pool = DB::Pool.new(ENV["DATABASE_URL"]) record User do field :id, PrimaryKey field :name, String field :email, String end user = User.new(name: "Alice", email: "alice@example.com") user.save # => true """ **Don't Do This:** Write raw SQL queries directly unless absolutely necessary for performance reasons or complex queries. ## 7. API Design and Versioning ### 7.1. Follow RESTful Principles **Do This:** Design APIs following RESTful principles. **Why:** RESTful APIs are easy to understand, predictable, and scalable. **Example:** * Use standard HTTP methods (GET, POST, PUT, DELETE). * Use nouns for resources (e.g., "/users", "/products"). * Use HTTP status codes to indicate success or failure. * Use hypermedia links (HATEOAS) to enable API discovery. **Don't Do This:** Create ad-hoc API designs that deviate from RESTful conventions. ### 7.2. API Versioning **Do This:** Use API versioning to maintain backward compatibility when introducing breaking changes. Use the header or URL versioning strategy **Why:** API versioning allows you to evolve your API without breaking existing clients. **Example (header versioning):** * Include a "version" parameter in the header of the request **Example (URL versioning):** * Prefix API endpoints with a version number (e.g., "/v1/users", "/v2/users"). **Don't Do This:** Make breaking API changes without versioning. Blindly update API without backward compatibility. ### 7.3. Documentation **Do This:** Document your APIs using tools like Swagger/OpenAPI. **Why:** API documentation makes it easier for developers to understand and use your APIs. **Example:** Use the openapi generator toolchain to automatically generate API clients and server stubs from OpenAPI specifications. **Don't Do This:** Neglect API documentation, or rely on incomplete or outdated documentation. ## 8. Community and Learning ### 8.1. Staying Up-to-Date **Do This:** Follow the official Crystal blog, release notes, and community forums to stay up-to-date with the latest developments in the language and ecosystem. Actively participate in the Crystal community on platforms like GitHub, Gitter, and Discourse to learn from others and contribute your knowledge. **Why:** Keeping current with the Crystal language and its evolving ecosystem ensures that you are using the latest features, best practices, and security updates. Community engagement provides opportunities for collaborative learning, knowledge sharing, and professional growth. **Don't Do This:** Rely solely on outdated tutorials or documentation. Avoid community discussions and feedback. ### 8.2 Contributing **Do This:** Consider contributing to open-source Crystal projects, whether by submitting bug fixes, suggesting new features, or improving documentation. Share your experience and knowledge with other developers through blog posts, articles, or conference talks. Promote Crystal within your organization and within the broader technology community. **Why:** Contributing to open-source projects helps improve the Crystal ecosystem for everyone. Sharing your knowledge and expertise fosters a culture of learning and innovation. Promoting Crystal helps to expand its user base and attract further investment in the language and its tooling. **Don't Do This:** Hoard your knowledge and contributions. Avoid participating in the Crystal community or promoting the language. This document provides a comprehensive set of tooling and ecosystem standards for Crystal development, which can significantly improve code quality, maintainability, and performance. By adhering to these guidelines, development teams can build robust and scalable applications.

# Security Best Practices Standards for Crystal This document outlines security best practices for Crystal development, serving as a guide for developers and AI coding assistants. It focuses on protecting against common vulnerabilities and implementing secure coding patterns specific to Crystal, embracing modern approaches based on the latest version. ## 1. Input Validation and Sanitization ### Standard Validate and sanitize all external input to prevent injection attacks (SQL injection, XSS, command injection). ### Why Untrusted data from users or external systems can corrupt data, execute malicious code, or compromise the system. ### Guidelines * **Do This:** * Always validate input data types, lengths, ranges, and formats against expected values. * Use prepared statements and parameterized queries for database interactions. * Encode output to prevent XSS vulnerabilities. * Use regular expressions or dedicated libraries for input validation. * **Don't Do This:** * Trust user input without validation. * Concatenate user input directly into SQL queries. * Display user-provided data directly without encoding. * Rely solely on client-side validation. ### Code Examples **SQL Injection Prevention (Prepared Statements)** """crystal require "db" DB.open("sqlite3:mydb.db") do |db| # Do This: Use prepared statements statement = db.prepare("SELECT * FROM users WHERE username = ? AND password = ?") username = "user1" password = "password123" result_set = statement.execute(username, password) result_set.each do |row| puts "User found: #{row}" end # Don't Do This: Avoid string interpolation # username = "user1; DROP TABLE users;" # Malicious input # query = "SELECT * FROM users WHERE username = '#{username}'" # Vulnerable! # db.query(query).each do |row| # puts "User found: #{row}" # end end """ **XSS Prevention (Output Encoding)** """crystal require "kemal" get "/" do |env| user_input = env.params["query"] || "Default Value" # Assume "query" parameter contains user input. # Do This: Escape HTML entities to prevent XSS escaped_input = CGI.escape_html(user_input) "<h1>You searched for: #{escaped_input}</h1>" end """ **Command Injection Prevention** """crystal # Do This: Use shell-safe methods or avoid shell commands whenever possible. require "process" def execute_safe_command(file_path : String) # Use Process::run to avoid shell interpretation of arguments result = Process.run("ls", [file_path]) if result.success? puts result.output else puts result.error end end # Example usage with a safe file path execute_safe_command("/safe/path/to/file.txt") # Don't Do This: Directly pass unsanitized user input to shell commands # def execute_command(user_input : String) # "ls #{user_input}" # Highly vulnerable to command injection # end # Example of VERY UNSAFE usage (demonstration purposes only!) # execute_command("$(rm -rf /)") # Example of malicious input """ ### Anti-Patterns * String interpolation directly in SQL queries. * Instead, always use prepared statements. * Trusting client-side validation without server-side verification. * Client-side validation is easily bypassed. * Displaying raw user input without encoding. * This can lead to XSS vulnerabilities. ## 2. Authentication and Authorization ### Standard Implement robust authentication and authorization mechanisms to identify and control user access. ### Why Weak authentication and authorization can lead to unauthorized access, data breaches, and compromised functionality. ### Guidelines * **Do This:** * Use strong password hashing algorithms (bcrypt, scrypt, argon2). * Implement multi-factor authentication (MFA). * Enforce principle of least privilege. * Use established authentication/authorization libraries (e.g., JWT, OAuth). * **Don't Do This:** * Store passwords in plain text. * Use weak hashing algorithms (MD5, SHA1). * Grant excessive permissions to users. * Implement custom authentication/authorization schemes without thorough security review. ### Code Examples **Password Hashing with bcrypt** """crystal require "bcrypt" def hash_password(password : String) : String BCrypt::Password.create(password) end def verify_password(password : String, hashed_password : String) : Bool BCrypt::Password.new(hashed_password) == password end # Example usage password = "mysecretpassword" hashed_password = hash_password(password) puts "Hashed password: #{hashed_password}" # Store the 'hashed_password' in database # Verification if verify_password("mysecretpassword", hashed_password) puts "Authentication successful!" else puts "Authentication failed!" end """ **JWT Authentication (using "jwt" shard)** """crystal require "jwt" require "time" SECRET_KEY = "your_secret_key" # Replace with a strong, randomly generated key def generate_token(user_id : Int32) : String payload = { "user_id" => user_id, "exp" => (Time.utc + 3600).to_unix # Token expiration time (1 hour) } JWT.encode(payload, SECRET_KEY, "HS256") end def verify_token(token : String) : Hash(String, JSON::Any)? begin decoded_token = JWT.decode(token, SECRET_KEY, true, {algorithm: "HS256"}) decoded_token[0].as_h? # Return the payload as a hash if verification is successful rescue JWT::DecodeError => e puts "Token verification failed: #{e.message}" nil end end # Example Usage user_id = 123 token = generate_token(user_id) puts "Generated token: #{token}" decoded = verify_token(token) if decoded puts "Authenticated User ID: #{decoded["user_id"]}" else puts "Authentication failed." end """ **Authorization (Role-Based Access Control)** """crystal enum Role Admin Editor Viewer end record User, id : Int32, role : Role def authorize(user : User, required_role : Role) : Bool user.role <= required_role end # Example Usage user = User.new(1, Role::Editor) if authorize(user, Role::Admin) puts "User is authorized to perform admin actions." elsif authorize(user, Role::Editor) puts "User is authorized to perform editor actions." else puts "User has insufficient privileges." end """ ### Anti-Patterns * Using predictable or easily guessable passwords. * Creating custom authentication schemes without rigorous security review. * Storing sensitive information in cookies without proper encryption and security flags. * Relying solely on front-end authorization. ## 3. Data Encryption and Storage ### Standard Encrypt sensitive data at rest and in transit to protect confidentiality. ### Why Data encryption prevents unauthorized access to sensitive information if systems are compromised. ### Guidelines * **Do This:** * Use TLS/SSL for all network communication. * Encrypt sensitive data at rest using strong encryption algorithms (AES). * Properly manage encryption keys and secrets. * Use secure storage solutions for sensitive data. * **Don't Do This:** * Transmit sensitive data over unencrypted channels (HTTP). * Store encryption keys in the code or version control. * Use weak or outdated encryption algorithms. ### Code Examples **TLS/SSL (using Kemal)** """crystal require "kemal" # Kemal framework defaults to TLS/SSL when running in production. # Ensure proper configuration for SSL certificates (e.g., using Let's Encrypt). # Configure Kemal to bind to HTTPS port (443 by default) and use your SSL certificates # In Kemal, ensure SSL certificates are properly configured via environment variables or command-line arguments. # KEMAL_SSL_CERT=path/to/your/certificate.pem # KEMAL_SSL_KEY=path/to/your/private_key.pem get "/" do |env| "Welcome to my secure site!" end """ **Data Encryption with AES (using "OpenSSL" binding)** """crystal require "openssl" KEY = "1234567890123456" # a 16-byte key for AES-128 IV = "abcdefghijklmnop" # Initialization Vector (must be 16 bytes for AES) def encrypt(data : String, key : String, iv : String) : String cipher = OpenSSL::Cipher.new("aes-128-cbc") cipher.encrypt cipher.key = key cipher.iv = iv encrypted = cipher.update(data) + cipher.final Base64.encode64(encrypted) end def decrypt(encrypted_data : String, key : String, iv : String) : String decipher = OpenSSL::Cipher.new("aes-128-cbc") decipher.decrypt decipher.key = key decipher.iv = iv decoded_data = Base64.decode64(encrypted_data) decrypted = decipher.update(decoded_data) + decipher.final decrypted end # Example Usage data = "Sensitive data to encrypt." encrypted_data = encrypt(data, KEY, IV) puts "Encrypted data: #{encrypted_data}" decrypted_data = decrypt(encrypted_data, KEY, IV) puts "Decrypted data: #{decrypted_data}" """ **Secure Secret Management (using environment variables)** """crystal # Do This: Use environment variables to store secrets db_password = ENV["DATABASE_PASSWORD"] if db_password.nil? puts "Error: DATABASE_PASSWORD environment variable not set!" exit(1) end puts "Connecting to database with password from environment." # Use db_password to connect to the database # DB.open("postgres://user:#{db_password}@host:port/database") do |db| ... end # Don't Do This: Hardcoding sensitive information or storing it in version control. #DB_PASSWORD = "hardcoded_password" # Insecure! """ ### Anti-Patterns * Storing encryption keys directly in the code. * Using weak or outdated encryption algorithms. * Transmitting sensitive data over unencrypted channels. * Failing to properly rotate encryption keys. ## 4. Error Handling and Logging ### Standard Implement proper error handling and logging mechanisms to identify and address security vulnerabilities. ### Why Detailed logs provide critical information for identifying and investigating security incidents. Careless error handling can leak sensitive information. ### Guidelines * **Do This:** * Log all security-related events (authentication failures, authorization violations, etc.). * Use structured logging formats (JSON) for easier analysis. * Implement centralized logging and monitoring. * Sanitize sensitive data before logging. * Handle exceptions gracefully and avoid revealing sensitive information in error messages. * **Don't Do This:** * Log sensitive information (passwords, API keys). * Use generic error messages that provide limited context. * Ignore exceptions without logging. ### Code Examples **Structured Logging** """crystal require "json" require "time" def log_event(level : String, message : String, data : Hash(String, String)? = nil) log_entry = { "timestamp" => Time.utc.to_s, "level" => level, "message" => message, "data" => data || {} } puts log_entry.to_json end # Example Usage log_event("INFO", "User logged in successfully", {"user_id" => "123", "ip_address" => "192.168.1.1"}) log_event("ERROR", "Authentication failed", {"username" => "testuser", "ip_address" => "192.168.1.1"}) # Sanitizing Sensitive Data sensitive_data = {"credit_card" => "1234-5678-9012-3456"} sanitized_data = {"credit_card" => "REDACTED"} log_event("WARN", "Suspicious activity detected", sanitized_data) # Log sanitized data """ **Error Handling** """crystal def safe_division(a : Int32, b : Int32) : Int32? begin a // b rescue DivisionByZero puts "Error: Division by zero!" log_event("ERROR", "Division by zero attempted", {"value_a" => a.to_s, "value_b" => b.to_s}) nil #Return nil to indicate failure end end # Example Usage result = safe_division(10, 0) if result puts "Result: #{result}" else puts "Operation failed." end def risky_operation(filename : String) begin file = File.open(filename) puts file.read_all file.close rescue e : Errno::ENOENT log_event("ERROR", "File not found", {"filename" => filename}) puts "File not found. Check logs for details." # User-friendly message rescue => e log_event("ERROR", "Generic error: #{e.message}", {"filename" => filename, "error" => e.class.name}) puts "An unexpected error occurred. Check logs for details." # Generic Message. end end risky_operation("non_existent_file.txt") #Example usage. """ ### Anti-Patterns * Logging sensitive information in plain text. * Using generic error messages that provide limited insight. * Over-logging, which can impact performance and storage. * Not having a central logging or alerting mechanism. ## 5. Dependencies and Vulnerability Management ### Standard Regularly update dependencies and scan for vulnerabilities to mitigate risks from third-party libraries. ### Why Vulnerabilities in dependencies can be exploited to compromise the application. ### Guidelines * **Do This:** * Use a dependency management tool (e.g., shards) to manage dependencies. * Regularly update dependencies to the latest versions. * Use security scanning tools to identify vulnerabilities in dependencies (e.g., "bundler-audit" equivalent). * Monitor security advisories for known vulnerabilities. * **Don't Do This:** * Use outdated or unsupported dependencies. * Ignore security advisories for dependencies. * Fail to regularly scan for vulnerabilities. ### Code Examples **Using Shards for Dependency Management** """yaml # shard.yml name: myapp version: 0.1.0 dependencies: kemal: github: kemalcr/kemal version: "~> 1.4" # Specify desired version. Use "~>" for compatible updates. jwt: github: joewandy/crystal-jwt version: "~> 0.5" license: MIT """ To install dependencies: """bash shards install """ To update dependencies to the latest compatible version: """bash shards update """ **Vulnerability Scanning Considerations** Crystal doesn't have a dedicated "bundler-audit" equivalent directly. Here are ways to approach this based on available information and tools. You can use these in combination for best results: 1. **Manual Review and Monitoring:** * Regularly check for security advisories related to the shards you are using. Monitor the GitHub repositories of your dependencies for reported issues and updates. * Subscribe to security mailing lists or RSS feeds that announce vulnerabilities in software libraries. 2. **Automated Vulnerability Scanning (Indirectly):** * **Container Scanning:** If you deploy your Crystal application within a Docker container, use container image scanning tools like Clair, Snyk, or Anchore to identify vulnerabilities in the base OS image and any system-level dependencies. * **CI/CD Integration:** Integrate security scanning into your CI/CD pipeline by running container scans during the build process. Fail the build if critical vulnerabilities are found. 3. **Shard-Specific Auditing (Future Tooling):** * The Crystal community may develop shard-specific auditing tools in the future. Keep an eye on community projects and announcements. Such tools would ideally analyze "shard.lock" and compare dependency versions against a known vulnerability database. **Example - Docker usage for scanning** """dockerfile FROM crystallang/crystal:latest WORKDIR /app COPY shard.yml shard.yml RUN shards install COPY . . # Dummy command to make the image scannable, replace with actual build/run execution CMD ["echo", "Application is built"] """ **Then use security scanners like "Trivy" to scan the docker image** """bash docker build -t my-crystal-app . trivy image my-crystal-app """ ### Anti-Patterns * Using outdated dependencies without updates. * Ignoring security advisories for dependencies. * Failing to scan for vulnerabilities in dependencies. * Vendoring dependencies without proper management. ## 6. Denial of Service (DoS) Protection ### Standard Implement measures to protect against denial-of-service (DoS) and distributed denial-of-service (DDoS) attacks. ### Why DoS attacks can overwhelm the system and make it unavailable to legitimate users. ### Guidelines * **Do This:** * Implement rate limiting to prevent abuse. * Use a web application firewall (WAF) to filter malicious traffic (Cloudflare, AWS WAF). * Implement request size limits. * Use caching to reduce server load. * **Don't Do This:** * Ignore potential DoS vectors. * Fail to monitor system resources for signs of attack. * Expose unnecessary endpoints. ### Code Examples **Rate Limiting (using Kemal middleware)** """crystal require "kemal" require "redis" # or any other caching system # Simple in-memory rate limiter (for demonstration; use Redis in production) RATE_LIMITS = {} of String => Int32 def rate_limit(env : HTTP::Env, redis : Redis) ip_address = env.remote_address.to_s key = "ratelimit:#{ip_address}" limit = 100 # requests per minute expiry = 60 # seconds # Atomic operation to increment the counter in Redis and get the value count = redis.incr(key) # Set expiry only if the key is new redis.expire(key, expiry) if count == 1 if count > limit env.response.status_code = 429 env.response.body = "Too Many Requests" env.response.headers["Retry-After"] = expiry.to_s return false # Halt the request else return true # Proceed with the request end end before do |env| redis = Redis.new("redis://127.0.0.1:6379") # Replace with your redis connection string. unless rate_limit(env, redis) halt # Stop processing the request if rate limit is exceeded end end get "/" do |env| "Hello, world!" end """ **Request Size Limits (Kemal)** """crystal require "kemal" #This configuration applies to all routes, can be configured at individual route levels. configure do |settings| settings.max_request_size = 10_000_000 # 10MB request limit end post "/upload" do |env| # By default Kemal already protects against large uploads due to the max_request_size configured. file = env.params.file("myfile") if file # Process the uploaded file puts "File name: #{file.filename}" puts "File size: #{file.tempfile.size}" "File uploaded successfully" else env.response.status_code = 400 "No file uploaded" end end """ ### Anti-Patterns * Not implementing any rate limiting. * Using overly permissive rate limits. * Failing to monitor for DoS attacks. * Relying solely on application-level defenses without network-level protection. ## 7. Session Management ### Standard Implement secure session management practices to protect user sessions from hijacking and unauthorized access. ### Why Vulnerable session management can lead to user account compromise. ### Guidelines * **Do This:** * Use strong, randomly generated session IDs * Regenerate the session ID after successful login * Set appropriate session expiration times * Store session data server-side * Use secure, HTTP-only cookies * Implement session timeout and idle timeout * Validate session data on each request * **Don't Do This:** * Use predictable session IDs * Store sensitive information in session cookies * Use excessively long session expiration times * Allow session fixation vulnerabilities. ### Code Examples **Secure Session Management (using Kemal middleware)** """crystal require "kemal" require "session" # configure session settings, use secure cookies, and use suitable secret. configure do |settings| settings.session_secret = "A_Very_Long_And_Random_Secret_Key" settings.session_store = Session::CookieStore.new( cookie_options: { secure: true, # Only transmit over HTTPS httponly: true, # Prevent client-side JavaScript access same_site: :strict # Helps prevent CSRF attacks } ) end before do |env| env.session # Accessing env.session initializes the session end get "/" do |env| session = env.session session["visits"] = (session["visits"] || 0).as(Int32) + 1 "Visits: #{session["visits"]}" end get "/login" do |env| # Simulating authentication successfully after submitted credentials via POST to /login env.session["user_id"] = 123 # Authenticated user ID # DO THIS: Regenerate session ID after login to prevent session fixation env.session.regenerate redirect "/" end get "/logout" do |env| env.session.clear # Clear all session data redirect "/" end """ ### Anti-Patterns * Using sequential or predictable session IDs. * Storing session IDs in the URL. * Using the same session ID for the entire session lifetime (without regeneration after login). * Using excessively long session lifetimes. * Not invalidating sessions on logout. * Using weak session secrets ## 8. File Uploads ### Standard Implement secure file upload handling to prevent malicious file uploads and potential security vulnerabilities. ### Why Unvalidated file uploads can lead to remote code execution, XSS, or denial of service attacks. ### Guidelines * **Do This:** * Validate file type based on content, not just the file extension * Sanitize filenames to prevent directory traversal attacks * Limit file sizes * Store uploaded files outside the web root * Use unique file names to prevent overwrites * Scan uploaded files for malware * Set restrictive permissions on uploaded files * **Don't Do This:** * Trust the file extension (e.g., "*.jpg", "*.png") * Store uploaded files directly in the web root * Allow arbitrary file names * Execute uploaded files directly * Use filename filtering as the only means of validation. ### Code Example """crystal require "kemal" PUBLIC_UPLOAD_DIR = "uploads" Dir.mkdir_p PUBLIC_UPLOAD_DIR # Create the directory if it doesn't exist post "/upload" do |env| file = env.params.file("myfile") if file filename = file.filename tempfile = file.tempfile # This gives you a file-like object # Validate File type based on content (MIME Type verification) instead of just extension mime_type = "file --mime-type -b #{tempfile.path}".strip # Requires 'file' utility allowed_mime_types = ["image/jpeg", "image/png", "image/gif"] unless allowed_mime_types.includes?(mime_type) env.response.status_code = 400 puts "Invalid file type: #{mime_type}" "Invalid file type." next end #Sanitize filename to prevent path traversal. sanitized_filename = filename.gsub(%r{[^a-zA-Z0-9._-]}) { "" } # Remove any characters that aren't letters, numbers, underscores, periods, or hyphens safe_filename = File.join(PUBLIC_UPLOAD_DIR, "#{Time.utc.to_unix}_#{sanitized_filename}") #Limit file uploads size. max_file_size = 10_000_000 #10MB if file.tempfile.size > max_file_size "File is too large" env.response.status_code = 413 next end # Save File File.copy(file.tempfile.path, safe_filename) puts "File uploaded to: #{safe_filename}" "File uploaded successfully." else env.response.status_code = 400 "No file uploaded" end end """ ### Anti-Patterns * Trusting client-provided filenames or MIME types without server-side validation. * Storing uploaded files in the webserver's document root without protecting them. * Allowing unrestricted file sizes or file types. * Not sanitizing uploaded filenames. * Attempting to execute uploaded files directly.

# Code Style and Conventions Standards for Crystal This document outlines the code style and conventions standards for the Crystal programming language. Adhering to these guidelines will promote code consistency, readability, and maintainability, and contribute significantly to collaborative development efforts. These guidelines are designed to align with the latest version of Crystal and leverage best practices in the Crystal ecosystem. The intention of this document is to also act as a contextual document that can be used in tandem with AI coding assistants such as GitHub Copilot. ## 1. General Formatting ### 1.1. Whitespace * **Do This:** Use 2 spaces for indentation. Avoid tabs. * **Don't Do This:** Use tabs or more/

# Component Design Standards for Crystal This document outlines component design standards for Crystal projects, aiming to foster reusable, maintainable, and performant code. These standards are tailored specifically for Crystal's features and ecosystem. ## 1. Component Architecture A component is a self-contained, reusable unit of functionality with a well-defined interface. In Crystal, components can be represented by modules, classes, structs, or even standalone functions. The overall application architecture should be component-based to promote modularity. ### 1.1. Loose Coupling * **Do This:** Design components with minimal knowledge of each other. Use interfaces or abstract classes to communicate. * **Don't Do This:** Create tight dependencies between components, making them difficult to change or reuse independently. * **Why:** Loose coupling enhances modularity and flexibility. Changes in one component are less likely to affect others. """crystal # Good: Using an interface abstract class PaymentProcessor abstract def process_payment(amount : Float64) end class StripeProcessor < PaymentProcessor def process_payment(amount : Float64) puts "Processing #{amount} with Stripe" end end class PaypalProcessor < PaymentProcessor def process_payment(amount : Float64) puts "Processing #{amount} with Paypal" end end # The client depends on the abstract PaymentProcessor, not the concrete implementations class Order def initialize(@payment_processor : PaymentProcessor) end def checkout(amount : Float64) @payment_processor.process_payment(amount) end end stripe = StripeProcessor.new order = Order.new(stripe) order.checkout(100.0) """ """crystal # Bad: Tightly Coupled class StripePayment def process(amount : Float64) puts "Processing payment using Stripe: #{amount}" end end class OrderProcessor def process_order(amount : Float64) stripe = StripePayment.new stripe.process(amount) # Tight coupling - OrderProcessor knows too much about StripePayment end end order_processor = OrderProcessor.new order_processor.process_order(50.0) """ ### 1.2. High Cohesion * **Do This:** Ensure each component encapsulates a single, well-defined purpose. * **Don't Do This:** Create God classes or modules that try to handle too many unrelated responsibilities. * **Why:** High cohesion leads to simpler, more understandable code. It simplifies maintenance and testing. """crystal # Good: High Cohesion module UserAuthentication def self.authenticate(username : String, password : String) : Bool # ... authentication logic ... true # Stubbed for example end def self.authorize(user_id : Int32, resource : String) : Bool # ... authorization logic ... true # Stubbed for example end end # The module is focused solely on user authentication and authorization """ """crystal # Bad: Low Cohesion module UserManagement def self.authenticate(username : String, password : String) : Bool # ... authentication logic ... true # Stubbed for example end def self.create_user(username : String, email : String) # ... user creation logic ... end def self.send_welcome_email(email : String) # ... send welcome email logic ... end end # The module attempts to handle authentication, user creation, and email sending - low cohesion! """ ### 1.3. Explicit Interfaces * **Do This:** Define clear interfaces for each component using abstract classes or modules with defined public methods. * **Don't Do This:** Rely on implicit interfaces or expose internal implementation details. * **Why:** Explicit interfaces promote predictability and allow for easier substitution of components. """crystal # Good: Interface definition using abstract class abstract class Storage abstract def save(key : String, value : String) abstract def load(key : String) : String? # String? to indicate nullable return abstract def delete(key : String) end class FileStorage < Storage def save(key : String, value : String) File.write(key, value) end def load(key : String) : String? File.read(key) rescue nil end def delete(key : String) File.delete(key) end end # Usage: storage = FileStorage.new storage.save("my_key", "my_value") puts storage.load("my_key") """ """crystal # Bad: Implicit Interface class DataWriter def write_data(data) # Implementation implies it takes any data puts "Writing data: #{data}" end end class DataProcessor def process(writer) writer.write_data({"key" => "value"}) # Assumes DataWriter has write_data, but no explicit contract. end end """ ## 2. Component Implementation ### 2.1. Use Crystal Generics Judiciously * **Do This:** Use generics to create reusable components that work with different types. * **Don't Do This:** Overuse generics, leading to complex and unreadable code. Consider type constraints. * **Why:** Generics enhance code reuse while maintaining type safety. """crystal # Good: Using Generics with Type Constraints class Cache(T) getter data : Hash(String, T) def initialize @data = Hash(String, T).new end def put(key : String, value : T) @data[key] = value end def get(key : String) : T? @data[key] end end cache_string = Cache(String).new cache_string.put("name", "Crystal") puts cache_string.get("name") cache_int = Cache(Int32).new cache_int.put("port", 8080) puts cache_int.get("port") """ """crystal # Bad: Overusing Generics class GenericProcessor(T, U, V) # Too many generic types, hard to reason about def process(input1 : T, input2 : U) : V # Complex logic involving T, U, and V... input1.to_s + input2.to_s # Just an example end end """ ### 2.2. Favor Composition over Inheritance * **Do This:** Build complex components by combining simpler components rather than relying heavily on inheritance. * **Don't Do This:** Create deep inheritance hierarchies that become rigid and difficult to maintain. * **Why:** Composition provides better flexibility and avoids the problems associated with the fragile base class problem. """crystal # Good: Composition example class Logger def log(message : String) puts "Log: #{message}" end end class StatsCollector def collect_stats(data) puts "Collecting Stats: #{data}" end end class Service def initialize(@logger : Logger, @stats : StatsCollector) end def run(data) @logger.log("Starting service") @stats.collect_stats(data) puts "Service running with #{data}" @logger.log("Service completed") end end logger = Logger.new stats = StatsCollector.new service = Service.new(logger, stats) service.run({"requests" => 100}) """ """crystal # Bad: Deep Inheritance class BaseComponent def initialize puts "Base Initialized" end end class ExtendedComponent < BaseComponent def initialize super # Always remember to call super! puts "Extended initialized" end end class EvenMoreExtended < ExtendedComponent def initialize super # Remember to call super! puts "EvenMoreExtended Initialized" end end component = EvenMoreExtended.new # Difficult to trace initialization flow. Inheritance makes debugging harder. """ ### 2.3. Use Modules for Namespaces and Utility Functions * **Do This:** Group related functions and constants within modules to avoid naming conflicts and organize code. * **Don't Do This:** Define global functions and constants that pollute the namespace. Modules offer a better approach. * **Why:** Modules create clear boundaries and improve code organization. """crystal # Good: Module Usage module StringUtils def self.reverse(str : String) : String str.reverse end def self.capitalize(str : String) : String str.capitalize end end puts StringUtils.reverse("hello") # olleh puts StringUtils.capitalize("world") # World """ """crystal # Bad: Global functions def reverse_string(str : String) : String str.reverse end def capitalize_string(str : String) : String str.capitalize end puts reverse_string("hello") puts capitalize_string("world") # pollute the global namespace """ ### 2.4. Immutability Where Possible * **Do This:** Favor immutable data structures and components to avoid unexpected side effects. * **Don't Do This:** Mutate data structures directly whenever possible. * **Why:** Immutability simplifies reasoning about code and improves thread safety. """crystal # Good: Immutability record Point, x : Int32, y : Int32 point1 = Point.new(1, 2) point2 = Point.new(point1.x + 1, point1.y + 1) # Creating new instance, not mutating puts point1 puts point2 """ """crystal # Bad: Mutable State class MutableCounter property count : Int32 def initialize @count = 0 end def increment @count += 1 # Mutates the state directly end end counter = MutableCounter.new counter.increment puts counter.count """ ### 2.5. Error Handling Strategy * **Do This:** Employ exceptions and "Result" types to deal with errors gracefully. * **Don't Do This:** Ignore errors or rely on return codes without proper handling. * **Why:** Robust error handling prevents application crashes and allows for graceful recovery. """crystal # Good: Using "Result" type for error handling require "option_parser" def safe_divide(a : Float64, b : Float64) : Result(Float64, String) if b == 0.0 return Error("Cannot divide by zero") else return Ok(a / b) end end result = safe_divide(10.0, 2.0) if result.success? puts "Result: #{result.value}" else puts "Error: #{result.error}" end result = safe_divide(5.0, 0.0) if result.success? puts "Result: #{result.value}" else puts "Error: #{result.error}" end """ """crystal # Bad: Ignoring Errors def divide(a : Float64, b : Float64) : Float64 a / b # No handling of potential division by zero end puts divide(10.0, 0.0) # Will crash the program """ ## 3. Specific Design Patterns for Crystal ### 3.1. Strategy Pattern * **How:** Define a family of algorithms, encapsulate each one, and make them interchangeable. Strategy lets the algorithm vary independently from clients that use it. """crystal # Strategy Interface abstract class CompressionStrategy abstract def compress(file_path : String) end # Concrete Strategies class ZipCompression < CompressionStrategy def compress(file_path : String) puts "Compressing #{file_path} using ZIP" end end class GzipCompression < CompressionStrategy def compress(file_path : String) puts "Compressing #{file_path} using GZIP" end end # Context class FileCompressor def initialize(@strategy : CompressionStrategy) end def compress_file(file_path : String) @strategy.compress(file_path) end end # Client Code zip_strategy = ZipCompression.new gzip_strategy = GzipCompression.new compressor = FileCompressor.new(zip_strategy) compressor.compress_file("my_file.txt") compressor = FileCompressor.new(gzip_strategy) compressor.compress_file("another_file.txt") """ ### 3.2. Observer Pattern * **How:** Define a one-to-many dependency between objects so that when one object changes state, all its dependents are notified and updated automatically. """crystal # Subject interface class Subject def initialize @observers = [] of Observer end def attach(observer : Observer) @observers << observer end def detach(observer : Observer) @observers.delete(observer) end def notify @observers.each { |observer| observer.update(self) } end end # Concrete Subject class NewsPublisher < Subject getter news : String def news=(news : String) @news = news notify # Notify after state change end end # Observer Interface abstract class Observer abstract def update(subject : Subject) end # Concrete Observers class EmailSubscriber < Observer def update(subject : Subject) puts "Email Subscriber received news: #{subject.news}" end end class SMSSubscriber < Observer def update(subject : Subject) puts "SMS Subscriber received news: #{subject.news}" end end # Client Code publisher = NewsPublisher.new email_subscriber = EmailSubscriber.new sms_subscriber = SMSSubscriber.new publisher.attach(email_subscriber) publisher.attach(sms_subscriber) publisher.news = "Crystal 1.7 released!" """ ### 3.3. Factory Pattern * **How:** Define an interface for creating an object, but let subclasses decide which class to instantiate. Factory Method lets a class defer instantiation to subclasses. """crystal # Abstract Product abstract class Document abstract def open end # Concrete Products class PDFDocument < Document def open puts "Opening PDF document" end end class TextDocument < Document def open puts "Opening Text document" end end # Abstract Factory abstract class DocumentFactory abstract def create_document : Document end # Concrete Factories class PDFFactory < DocumentFactory def create_document : Document PDFDocument.new end end class TextFactory < DocumentFactory def create_document : Document TextDocument.new end end # Client Code pdf_factory = PDFFactory.new pdf_document = pdf_factory.create_document pdf_document.open text_factory = TextFactory.new text_document = text_factory.create_document text_document.open """ ## 4. Code Style and Formatting * **Use "crystal tool format":** Consistently format code using the built-in formatter. * **Follow Naming Conventions:** class "PascalCase", method "snake_case". * **Limit Line Length:** Aim for a maximum of 80-120 characters per line. * **Add Comments:** Clear and concise comments to explain complex logic. Use proper documentation for public APIs. ## 5. Conclusion Adhering to these component design standards will significantly improve the quality, maintainability, and reusability of Crystal codebases. This document will serve as a guide for new developers and a reference for maintaining consistent code quality across projects.

# Deployment and DevOps Standards for Crystal This document outlines the coding standards for deployment and DevOps practices in Crystal. It aims to guide developers in building reliable, scalable, and maintainable Crystal applications. These standards should be used with AI-assisted coding tools to improve the quality and consistency of the code produced. ## 1. Build Processes and CI/CD ### 1.1 General Principles * **Consistent Builds:** Ensure that builds are reproducible across different environments. * **Automated Pipelines:** Automate build, test, and deployment processes. * **Infrastructure as Code (IaC):** Manage infrastructure using code to enhance repeatability and traceability. ### 1.2 Standards * **Do This:** Use a build tool like "make" or "crystal Makefile". * **Why:** Provides a standard interface for building and managing your application. """makefile .PHONY: all deps build test release clean PREFIX ?= /usr/local all: deps build deps: shards install build: crystal build src/your_app.cr --release test: crystal spec install: build install -D -m 755 ./your_app $(PREFIX)/bin/your_app uninstall: rm -f $(PREFIX)/bin/your_app release: build tar czvf your_app-$(VERSION).tar.gz your_app config/application.yml clean: rm -rf ./bin/* ./lib/* .shards/ """ * **Don't Do This:** Manually compile and deploy your application. * **Why:** Increases the risk of inconsistencies and errors. ### 1.3 CI/CD Pipeline * **Do This:** Use CI/CD tools like GitHub Actions, GitLab CI, Jenkins, or CircleCI. * **Why:** Automates build, test, and deployment processes, improving efficiency and reducing errors. """yaml # .github/workflows/main.yml name: CI/CD on: push: branches: [ main ] pull_request: branches: [ main ] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Install Crystal uses: oprypin/install-crystal@v1 with: crystal: 1.11.2 # Use a specific, recent stable version - name: Install Dependencies run: shards install - name: Build run: crystal build src/your_app.cr --release - name: Run Tests run: crystal spec - name: Create Release Archive if: github.ref == 'refs/heads/main' && github.event_name == 'push' run: | VERSION=$(git describe --tags --abbrev=0) tar czvf your_app-${VERSION}.tar.gz your_app config/application.yml mv your_app-${VERSION}.tar.gz release.tar.gz - name: Upload Release Archive if: github.ref == 'refs/heads/main' && github.event_name == 'push' uses: actions/upload-artifact@v3 with: name: release-archive path: release.tar.gz """ * **Don't Do This:** Deploy directly from your development machine. * **Why:** Introduces inconsistencies and potential security risks. ### 1.4 Containerization * **Do This:** Use Docker to build and deploy your Crystal application. * **Why:** Ensures consistent environments across different stages (development, testing, production). """dockerfile # Dockerfile FROM crystallang/crystal:latest-alpine as builder WORKDIR /app # Copy only the shard.yml and shard.lock to leverage Docker cache COPY shard.yml shard.lock ./ # Install shards dependencies RUN shards install --production COPY . . RUN crystal build src/your_app.cr --release FROM alpine:latest WORKDIR /app COPY --from=builder /app/your_app ./ COPY config/application.yml ./ EXPOSE 3000 CMD ["./your_app"] """ * **Don't Do This:** Deploy your application directly without containerization. * **Why:** Creates dependency conflicts and inconsistencies. ### 1.5 Infrastructure as Code (IaC) * **Do This:** Use tools like Terraform, Ansible, or Chef to manage infrastructure. * **Why:** Infrastructure can be versioned and managed more effectively. """terraform # Terraform example (simplified) resource "aws_instance" "app_server" { ami = "ami-xxxxxxxxxxxxxxxxx" # Replace with a valid AMI instance_type = "t2.micro" tags = { Name = "your_app-server" } } output "public_ip" { value = aws_instance.app_server.public_ip } """ * **Don't Do This:** Manually provision and configure servers. * **Why:** Error-prone and difficult to maintain. ## 2. Production Considerations ### 2.1 General Principles * **Configuration Management:** Separate configuration from code. * **Logging and Monitoring:** Implement comprehensive logging and monitoring. * **Security Hardening:** Follow security best practices. ### 2.2 Configuration Management * **Do This:** Use environment variables or configuration files (e.g., YAML, JSON) to manage application settings. * **Why:** Allows you to change settings without modifying the code. """crystal require "yaml" CONFIG_PATH = "./config/application.yml" CONFIG = YAML.parse_file(CONFIG_PATH)[ENV["CRYSTAL_ENV"]? || "development"].as_h DB_HOST = CONFIG["db_host"].as_s DB_PORT = CONFIG["db_port"].as_i """ """yaml # config/application.yml development: db_host: "localhost" db_port: 5432 production: db_host: "db.example.com" db_port: 5432 """ * **Don't Do This:** Hardcode configuration values in your application. * **Why:** Makes it difficult to manage settings across different environments. ### 2.3 Logging * **Do This:** Use a logging library like "log" or "kemal-logger" and log structured data. * **Why:** Easier to debug issues and monitor application health. """crystal require "log" Log.setup(level: Log::Severity::INFO) Log.info { "Application started" } begin # Some code that might raise an exception raise "Something went wrong" rescue e : Exception Log.error(exception: e, message: "An error occurred") end """ * **Don't Do This:** Rely solely on "puts" or "print" statements for logging. * **Why:** Difficult to manage and analyze logs in a production environment. ### 2.4 Monitoring * **Do This:** Integrate with monitoring tools like Prometheus, Grafana, DataDog, or New Relic. Use metrics and alerts. * **Why:** Gives insight into application performance and health. """crystal # Example with Prometheus (using a hypothetical client) require "prometheus" metrics = Prometheus::Registry.new http_requests_total = metrics.counter( :http_requests_total, "Total number of HTTP requests." ) get "/metrics" do |context| context.response.content_type = "text/plain; version=0.0.4; charset=utf-8" context.response.print metrics.to_text end before_all do |context| http_requests_total.increment end """ * **Don't Do This:** Neglect to monitor your application in production. * **Why:** You won't be aware of performance issues or errors. ### 2.5 Security * **Do This:** * Follow security best practices for web applications (e.g., OWASP). * Use HTTPS for all communication. * Regularly update dependencies to patch security vulnerabilities. * Implement input validation and output encoding & escaping to prevent injection attacks. * **Don't Do This:** * Store sensitive information (e.g., passwords, API keys) in code. * Expose unnecessary ports or services. * Run your application as root. ### 2.6 Handling Secrets * **Do This:** Utilize secret management tools like Vault, AWS Secrets Manager, or environment variables in secure deployment systems. * **Why:** Protect sensitive information from unauthorized access. """crystal # Example using environment variables (ensure the deployment platform supports masking sensitive env vars) DB_PASSWORD = ENV["DB_PASSWORD"] unless DB_PASSWORD Log.fatal { "DB_PASSWORD environment variable not set!" } exit(1) end # Use DB_PASSWORD to connect to the database """ * **Don't Do This:** Store secrets directly in configuration files or environment variables exposed within the code repository. * **Why:** Compromises the security of your application. ### 2.7 Zero-Downtime Deployments * **Do This:** Implement rolling deployments or blue/green deployments to minimize downtime. * **Why:** Ensures continuous availability of your application. * **How:** Uses Kubernetes deployments strategies. """yaml # Kubernetes Deployment example (rolling update) apiVersion: apps/v1 kind: Deployment metadata: name: your-app-deployment spec: replicas: 3 selector: matchLabels: app: your-app strategy: type: RollingUpdate rollingUpdate: maxSurge: 25% maxUnavailable: 25% template: metadata: labels: app: your-app spec: containers: - name: your-app-container image: your-docker-registry/your-app:latest # tag updates automatically ports: - containerPort: 3000 """ * **Don't Do This:** Deploy your application without a proper deployment strategy resulting in downtime. * **Why:** disrupts user experience and potentially causes loss of revenue. ### 2.8 Health Checks * **Do This:** Implement health check endpoints for your application. * **Why:** Allows load balancers and orchestration tools (like Kubernetes) to monitor the health of your application instances. """crystal require "kemal" get "/health" do |context| context.response.status_code = 200 context.response.print "OK" end Kemal.run """ * **Don't Do This:** Deploy without a health check. * **Why:** Makes automatic recovery and scaling difficult. ## 3. Performance Optimization ### 3.1 General Principles * **Profiling:** Identify performance bottlenecks. * **Caching:** Implement caching strategies. * **Resource Limits:** Define memory and CPU limits. ### 3.2 Profiling * **Do This:** Use tools like "perf" or "flamegraph" to profile your application. * **Why:** Helps you identify and address performance bottlenecks. """bash # Example profiling with perf perf record -g ./your_app perf report """ * **Don't Do This:** Make performance optimizations without profiling first. * **Why:** Wastes time and effort on potentially insignificant issues. ### 3.3 Caching * **Do This:** Implement caching using solutions like Redis or Memcached. * **Why:** Reduces database load and improves response times. """crystal require "redis" redis = Redis.new(host: "localhost", port: 6379) def get_data(key : String) cached_data = redis.get(key) if cached_data return cached_data else # Fetch data from the database data = fetch_from_db(key) redis.setex(key, 3600, data) # Cache for one hour return data end end """ * **Don't Do This:** Cache data indefinitely without invalidation. * **Why:** Can lead to stale data and incorrect results. ### 3.4 Resource Limits * **Do This:** Set memory and CPU limits for your containers or processes. * **Why:** Prevents resource exhaustion and improves stability. """yaml # Kubernetes Pod example with resource limits apiVersion: v1 kind: Pod metadata: name: your-app-pod spec: containers: - name: your-app-container image: your-docker-registry/your-app:latest resources: limits: memory: "512Mi" cpu: "1" requests: memory: "256Mi" cpu: "0.5" """ * **Don't Do This:** Fail to set resource limits. * **Why:** Increases the risk of resource depletion and application crashes. ### 3.5 Cluster Management * **Do This:** Create a cluster using Kubernetes, ECS, or similar solutions. * **Why:** Allows you to scale your app quickly, and reliably. """yaml # Sample Kubernetes Deployment manifest for scaling apiVersion: apps/v1 kind: Deployment metadata: name: crystal-app spec: replicas: 3 # Start with 3 replicas selector: matchLabels: app: crystal-app template: metadata: labels: app: crystal-app spec: containers: - name: crystal-app image: your-repo/crystal-app:latest ports: - containerPort: 8080 """ **Explanation:** This Deployment manifest creates three replicas, allowing you to distribute traffic. You could scale up or down by altering the replication factor. * **Don't Do This:** Have a single point of failure for your application. * **Why:** A cluster ensures higher availability and better scalability. ## 4. Database Management ### 4.1 General Principles * **Connection Pooling:** Reuse database connections. * **Migrations:** Manage database schema changes. * **Backups:** Implement regular database backups. ### 4.2 Connection Pooling * **Do This:** Use a connection pool to manage database connections. * **Why:** Reduces the overhead of establishing new connections for each request. """crystal require "pg" pool = PG::Pool.new(host: "localhost", port: 5432, user: "username", password: "password", database: "mydb", max_connections: 10) def query_db(sql : String) pool.use do |connection| result = connection.exec(sql) # Process the result end end """ * **Don't Do This:** Create a new database connection for each request. * **Why:** Inefficient and can exhaust database resources. ### 4.3 Migrations * **Do This:** Use a migration tool like "migrate" or implement your own migration system. * **Why:** Allows you to manage database schema changes in a controlled and versioned manner. """crystal # Example migration using a hypothetical migration library require "my_migration_lib" migration = MyMigrationLib::Migration.new migration.up do execute("CREATE TABLE users (id SERIAL PRIMARY KEY, name VARCHAR(255))") end migration.down do execute("DROP TABLE users") end """ * **Don't Do This:** Manually alter the database schema without a migration system. * **Why:** Difficult to track changes and revert if necessary. ### 4.4 Backups * **Do This:** Implement regular database backups. * **Why:** Protects your data from loss in case of hardware failures or other disasters. """bash # Example database backup script pg_dump -h localhost -p 5432 -U username -d mydb -f backup.sql """ * **Don't Do This:** Neglect to back up your database. * **Why:** Could lead to permanent data loss. ## 5. Conclusion Adhering to these DevOps standards will help you build robust, scalable, and maintainable Crystal applications. By automating build processes, implementing comprehensive logging and monitoring, and following security best practices, you can ensure the reliability and performance of your applications in production. Continue to iterate and refine these standards as Crystal evolves and new best practices emerge.