Cline

# Testing Methodologies Standards for Hetzner This document outlines the testing methodologies standards for Hetzner infrastructure and application deployment. It is designed to guide developers in ensuring the reliability, performance, and security of solutions built on the Hetzner Cloud platform. These standards will serve as context for AI coding assistants, promoting consistency and quality across all projects. ## 1. Introduction to Testing in Hetzner High-quality testing is crucial for building robust and reliable applications in the Hetzner ecosystem. Due to the distributed nature of cloud deployments, a comprehensive testing strategy is vital. This standard establishes guidelines for effectively testing applications and infrastructure deployed on Hetzner Cloud. It covers unit tests, integration tests, end-to-end tests, and considerations specific to Hetzner’s features and services. ## 2. Unit Testing ### 2.1 Standard: Isolate Units for Testing **Do This:** Write unit tests that isolate individual components (functions, classes, or modules) to verify their behavior in isolation. Use mocking or stubbing to replace dependencies. **Don't Do This:** Avoid writing unit tests that rely on external services or databases, as this makes them slow, unreliable, and difficult to maintain. Never skip unit tests for critical modules due to perceived complexity. **Why This Matters:** Unit tests are the fastest and most reliable form of testing. They help detect errors early in the development cycle, making them easier and cheaper to fix. **Code Example (Python):** """python # app/calculator.py class Calculator: def add(self, a, b): return a + b # tests/test_calculator.py import unittest from unittest.mock import MagicMock from app.calculator import Calculator class TestCalculator(unittest.TestCase): def test_add(self): calc = Calculator() result = calc.add(2, 3) self.assertEqual(result, 5) """ **Anti-Pattern:** Writing unit tests that interact with database connections or Hetzner Cloud APIs directly leads to flaky tests. **Hetzner Specific Detail:** When using Hetzner's Load Balancers, ensure unit tests for backend services stub out the load balancer interaction. ### 2.2 Standard: Test All Code Paths **Do This:** Aim for high test coverage by writing tests that exercise all possible execution paths within a unit. Cover all conditional branches, loops, and error handling logic. **Don't Do This:** Do not ignore edge cases or error scenarios. A test suite with low coverage can give a false sense of security. **Why This Matters:** Comprehensive test coverage helps ensure that all parts of the code are working correctly, even under unusual circumstances. **Code Example (Go):** """go // calculator.go package calculator import "errors" func Divide(a, b float64) (float64, error) { if b == 0 { return 0, errors.New("cannot divide by zero") } return a / b, nil } // calculator_test.go package calculator import "testing" func TestDivide(t *testing.T) { t.Run("Valid Division", func(t *testing.T) { result, err := Divide(10, 2) if err != nil { t.Errorf("Expected no error, got %v", err) } if result != 5 { t.Errorf("Expected 5, got %f", result) } }) t.Run("Divide by Zero", func(t *testing.T) { _, err := Divide(10, 0) if err == nil { t.Errorf("Expected an error, got nil") } }) } """ **Anti-Pattern:** Only testing happy path scenarios and neglecting error conditions and edge cases. **Hetzner Specific Detail:** If using Hetzner's Object Storage, ensure that unit tests for code interacting with the storage service mock the API responses properly. ### 2.3 Standard: Assertions and Expected Outcomes **Do This:** Use clear and specific assertions to verify that the code behaves as expected. Ensure that assertions are easy to understand and provide meaningful error messages when they fail. Use descriptive test names. **Don't Do This:** Avoid using generic assertions that only check for the absence of errors. Don't use vague test names that don't explain what's supposed to happen. **Why This Matters:** Clear assertions make it easier to diagnose and fix problems when tests fail. **Code Example (JavaScript with Jest):** """javascript // calculator.js function multiply(a, b) { return a * b; } // calculator.test.js test('multiplies two numbers correctly', () => { expect(multiply(2, 3)).toBe(6); }); """ **Anti-Pattern:** Writing tests with assertions that do not clearly define expected outcomes. ## 3. Integration Testing ### 3.1 Standard: Verify Interactions Between Components **Do This:** Write integration tests to verify that different components of the application work together correctly. Focus on testing the interactions between modules, services, and external systems. Use test containers to simulate the environment. **Don't Do This:** Avoid writing integration tests that are too broad or too narrow. Don't create integration tests with a massive scope, making debugging difficult. **Why This Matters:** Integration tests ensure that the different parts of the application are compatible and work seamlessly together. **Code Example (Docker Compose and Python):** *docker-compose.yml* """yaml version: "3.8" services: app: build: . ports: - "5000:5000" depends_on: - db environment: - DATABASE_URL=postgresql://user:password@db:5432/app_db db: image: postgres:13 environment: - POSTGRES_USER=user - POSTGRES_PASSWORD=password - POSTGRES_DB=app_db """ *test\_integration.py* """python import unittest import requests import os class TestIntegration(unittest.TestCase): def setUp(self): self.app_url = "http://localhost:5000" # Assuming app runs on port 5000 def test_app_db_connection(self): try: response = requests.get(f"{self.app_url}/health") # Using a /health endpoint example. Add relevant test endpoint. response.raise_for_status() # Raises HTTPError for bad responses (4xx or 5xx) self.assertEqual(response.status_code, 200) except requests.exceptions.RequestException as e: self.fail(f"Request failed: {e}") """ **Anti-Pattern:** Skipping integration tests due to complexity in setting up the environment often leads to issues in production. **Hetzner Specific Detail:** For applications leveraging Hetzner's managed databases, integration tests should verify connectivity and data integrity within the Hetzner environment. Use "testcontainers" or similiar libraries for containerized integration tests. ### 3.2 Standard: Test Data Management **Do This:** Manage test data carefully to ensure that tests are repeatable and reliable. Use dedicated test databases or schemas to avoid polluting production data. Clean up test data after each test run. **Don't Do This:** Avoid using hardcoded test data or relying on shared test data that can be modified by other tests. Do not skip data cleanup, causing test pollution. **Why This Matters:** Proper test data management prevents tests from interfering with each other and ensures that tests produce consistent results. **Code Example (SQLAlchemy and Python):** """python # test_data_manager.py from sqlalchemy import create_engine from sqlalchemy.orm import sessionmaker from your_app import Base # Imports Base from your application DATABASE_URL = "postgresql://test_user:test_password@localhost:5432/test_db" engine = create_engine(DATABASE_URL) TestingSessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine) def create_test_database(): Base.metadata.create_all(bind=engine) def drop_test_database(): Base.metadata.drop_all(bind=engine) def get_test_db(): db = TestingSessionLocal() try: yield db finally: db.close() """ """python # your_test.py import unittest from test_data_manager import create_test_database, drop_test_database, get_test_db from your_app import YourModel # Import app's model class YourTest(unittest.TestCase): @classmethod def setUpClass(cls): create_test_database() # Setup the test database once for the class @classmethod def tearDownClass(cls): drop_test_database() # Drop the test database after tests are done def setUp(self): self.db = next(get_test_db()) # Get a fresh DB session for each test # Add any default data needed for tests each time self.db.add(YourModel(name="Test Data")) self.db.commit() def tearDown(self): self.db.rollback() # Rollback any transaction made in test self.db.query(YourModel).delete() # Clear out the data self.db.commit() self.db.close() # Close the DB session def test_your_endpoint(self): # Test Logic here pass """ **Hetzner Specific Detail:** When running integration tests against resources on Hetzner Cloud, use unique naming conventions or dedicated testing resources to avoid conflicts with production environments. Always use environment variables instead of hardcoding credentials. For example: """bash export HCLOUD_TOKEN="your_hetzner_cloud_token" """ ### 3.3 Standard: Test Dependencies Using Service Containers **Do This:** Utilize containerization, such as Docker, to manage and isolate test dependencies. This creates a reproducible test environment. Ensure proper configuration is set up for each container. **Don't Do This:** Relying on system-wide dependencies for integration tests that are not containerized, as this can lead to inconsistent and unreliable results. **Why This Matters:** Service containers ensure a consistent and isolated testing environment, minimizing the risk of environment-specific issues. **Code Example (Using Docker Compose for PostgreSQL):** """yaml version: "3.8" services: db: image: postgres:13 ports: - "5432:5432" environment: POSTGRES_USER: test_user POSTGRES_PASSWORD: test_password POSTGRES_DB: test_db """ Then, within your integration tests: """python import psycopg2 def test_database_connection(): try: conn = psycopg2.connect( host="localhost", # Or the Docker Compose service name "db" if running within the same network port=5432, user="test_user", password="test_password", database="test_db" ) cursor = conn.cursor() cursor.execute("SELECT 1") result = cursor.fetchone() assert result[0] == 1 except psycopg2.Error as e: assert False, f"Database connection failed: {e}" finally: if conn: cursor.close() conn.close() """ **Anti-Pattern:** Running integration tests that depend on manually installed services often results in brittle and unreliable execution. ## 4. End-to-End Testing ### 4.1 Standard: Simulate Real User Scenarios **Do This:** Write end-to-end (E2E) tests to simulate real user interactions with the application. Design E2E tests that cover the most critical user flows, such as registration, login, and common tasks. **Don't Do This:** Avoid writing E2E tests that are too granular or too complex. Overly complex E2E tests are hard to maintain and debug. Don't cover scenarios that are already covered by unit or integration testing. **Why This Matters:** E2E tests ensure that the application works correctly from the user's perspective, including all the different layers and dependencies. **Code Example (Selenium with Python):** """python from selenium import webdriver from selenium.webdriver.common.by import By import unittest class TestE2E(unittest.TestCase): def setUp(self): self.driver = webdriver.Chrome() # Ensure ChromeDriver is installed and in PATH self.driver.get("http://localhost:8000") # Replace with your app's URL def tearDown(self): self.driver.quit() def test_login(self): username_input = self.driver.find_element(By.ID, "username") password_input = self.driver.find_element(By.ID, "password") login_button = self.driver.find_element(By.ID, "login-button") username_input.send_keys("testuser") password_input.send_keys("password123") login_button.click() # Assert that we logged in successfully self.assertEqual(self.driver.current_url, "http://localhost:8000/dashboard") """ **Anti-Pattern:** Skimping on E2E tests due to their overhead and complexity, as these tests are crucial for verifying the entire system's functionality. **Hetzner Specific Detail:** If an application interacts with Hetzner Cloud services from the frontend, simulate these interactions in E2E tests, such as creating a server or modifying firewall rules. Ensure you clean up these resources after the test runs. ### 4.2 Standard: Test Environment Parity **Do This:** Use a test environment that closely mirrors the production environment, including the same operating system, web server, database, and dependencies. Configure the test environment to use realistic data volumes and network configurations. **Don't Do This:** Do not test in an environment that is significantly different from production, as this can lead to false positives and false negatives. Also, avoid using simplified or scaled-down environments for E2E testing. **Why This Matters:** Environment parity ensures that the tests accurately reflect the behavior of the application in production. **Code Example (Terraform for environment creation):** """terraform # main.tf resource "hcloud_server" "test_server" { name = "e2e-test-server" server_type = "cx11" image = "ubuntu-22.04" location = "fsn1" # Change to your desired location } # outputs.tf output "server_ip" { value = hcloud_server.test_server.ipv4_address } """ **Anti-Pattern:** Testing on local development machines and not validating the solution on a production-like environment hosted on Hetzner. ### 4.3 Standard: Monitoring and Logging during tests **Do This:** Implement comprehensive monitoring and logging during E2E test execution. Analyze logs and metrics to identify performance bottlenecks and errors. **Don't Do This:** Do not run E2E tests without proper monitoring, making it difficult to pinpoint the root cause of a failure. Don't ignore unusual performance metrics during testing. **Why This Matters:** Monitoring and logging enable you to identify and diagnose problems that may not be immediately apparent from the test results. **Code Example (Integrating Prometheus and Grafana):** 1. **Install Prometheus and Grafana:** Deploy Prometheus and Grafana on Hetzner Cloud (e.g., using Docker). 2. **Configure Prometheus to scrape the application:** Expose application metrics (e.g., using Micrometer in Java or Prometheus client library for Python). 3. **Create Grafana dashboards:** Visualize key performance indicators (KPIs) during E2E test runs. **Hetzner Specific Detail:** Leverage Hetzner's metrics exporter to gather and analyze server-level metrics, such as CPU usage, memory consumption, and network traffic, during E2E testing. ## 5. Security Testing ### 5.1 Standard: Security Scanning **Do This:** Use automated security scanning tools to identify vulnerabilities in the application and infrastructure. Integrate security scanning into the CI/CD pipeline. Test for common security flaws. Integrate tools like Clair, Trivy, or custom scripts. **Don't Do This:** Do not rely solely on manual security reviews. Do not skip security scanning due to time constraints. **Why This Matters:** Automated security scanning helps detect and remediate vulnerabilities before they can be exploited. **Code Example (Using Trivy in a CI/CD pipeline):** """yaml # .gitlab-ci.yml stages: - build - test build: stage: build image: docker:latest services: - docker:dind script: - docker build -t my-app . - docker save -o my-app.tar my-app artifacts: paths: - my-app.tar security_scan: stage: test image: aquasec/trivy:latest script: - trivy image --severity HIGH,CRITICAL --exit-code 0 my-app """ ### 5.2 Standard: Authentication and Authorization **Do This:** Thoroughly test authentication and authorization mechanisms to ensure that users can only access the resources they are authorized to access. Test for common vulnerabilities such as broken authentication, authorization flaws, and session management issues. **Don't Do This:** Do not use default credentials. Do not rely on client-side validation for security. **Why This Matters:** Authentication and authorization are critical for protecting sensitive data and preventing unauthorized access. **Code Example (Testing authentication):** """python import requests import unittest class TestAuth(unittest.TestCase): def test_login_success(self): data = {'username': 'valid_user', 'password': 'valid_password'} r = requests.post('http://localhost:5000/login', data=data) # Replace URL self.assertEqual(r.status_code, 200) #Check HTTP Status def test_login_fail(self): data = {'username': 'valid_user', 'password': 'wrong_password'} r = requests.post('http://localhost:5000/login', data=data) # Replace URL self.assertEqual(r.status_code, 401) #Check HTTP Status """ **Hetzner Specific Detail:** When using Hetzner's firewall, regularly test firewall rules to ensure they are correctly configured and prevent unauthorized access to servers. ### 5.3 Standard: Data Validation **Do This:** Validate all input data to prevent injection attacks (SQL injection, cross-site scripting). Use parameterized queries or ORMs to prevent SQL injection, and sanitize user input to prevent XSS attacks. **Don't Do This:** Do not trust user input. Do not disable input validation. **Why This Matters:** Data validation is essential for preventing a wide range of security vulnerabilities. ## 6. Performance Testing ### 6.1 Standard: Load Testing **Do This:** Perform load testing to evaluate the application's behavior under heavy load. Identify performance bottlenecks and ensure that the application can handle the expected traffic volume. Use tools like Locust, JMeter, or k6.io. **Don't Do This:** Do not deploy applications without load testing. Do not underestimate expected traffic volume. **Why This Matters:** Load testing helps ensure that the application can handle peak traffic and prevent performance degradation. **Code Example (Locustfile):** """python from locust import HttpUser, task, between class QuickstartUser(HttpUser): wait_time = between(1, 2) @task def hello_world(self): self.client.get("/hello") # Define test endpoint """ ### 6.2 Standard: Monitoring Application Performance **Do This:** Instrument applications with tools that can expose metrics for monitoring performance. Tools like Prometheus, Grafana, and the ELK stack help track application performance in real-time. **Don't Do This:** Ignore performance monitoring, as this is crucial for identifying and addressing bottlenecks. **Why This Matters:** Monitoring helps identify performance issues and allows for proactive optimization. **Hetzner Specific Detail:** Leverage Hetzner Cloud's monitoring features to track server performance metrics, such as CPU usage, memory consumption, and network traffic. ### 6.3 Standard: Scalability Testing **Do This:** Verify application scalability by testing its ability to handle increased load by adding resources. This includes horizontal (adding more servers) and vertical (increasing server resources) scaling. **Don't Do This:** Neglect to test scalability, as the long-term implications can be detrimental. ## 7. Automation and CI/CD ### 7.1 Standard: Integrate Testing into CI/CD **Do This:** Integrate all types of tests into the CI/CD pipeline to ensure that code changes are automatically tested before being deployed to production. Implement automated deployment pipelines to Hetzner Cloud. **Don't Do This:** Deploy code without automated testing. Manually run tests, as this introduces delays and inconsistencies. **Why This Matters:** Automated testing in the CI/CD pipeline ensures that code changes are thoroughly tested, reducing the risk of introducing bugs into production. **Code Example (GitLab CI with integration tests):** """yaml stages: - test integration_tests: image: python:3.9 stage: test services: - postgres:13 variables: DATABASE_URL: postgresql://user:password@postgres:5432/testdb before_script: - pip install -r requirements.txt script: - python -m unittest discover -s tests -p "test_integration*.py" """ ### 7.2 Standard: Test Environment Provisioning **Do This:** Automate the provisioning and configuration of test environments using infrastructure-as-code tools like Terraform. Ensure all necessary services and dependencies are automatically set up for each test run. **Don't Do This:** Manually set up test environments, resulting in inconsistencies and increased setup time. **Why This Matters:** Automated test environment provisioning ensures that test environments are consistent and repeatable, reducing the risk of environment-specific issues. ### 7.3 Standard: Infrastructure Testing **Do This:** Implement infrastructure tests to validate that infrastructure components are correctly provisioned and configured. Utilize tools like Terraform for infrastructure testing. ## 8. Documentation and Reporting ### 8.1 Standard: Document Test Plans and Results **Do This:** Document the test plan, including the scope, objectives, and test cases. Provide clear, detailed reports of test results. **Don't Do This:** Skip test documentation, as this hinders the ability to track progress, analyze results, and make informed decisions about code quality. Don't generate ambiguous or incomplete reports. **Why This Matters:** Detailed documentation facilitates collaboration, knowledge sharing, and continuous improvement of the testing process. ### 8.2 Standard: Centralized Test Reporting **Do This:** Centralize test reporting using tools like TestRail, Xray, or similar solutions to track test execution results and identify trends, metrics, and areas for improvement. ### 8.3 Standard: Communicating Results **Do This:** Create clear, concise reports that are easily understood by all stakeholders. Communicate with the development team to rapidly address test failures. ## 9. Hetzner Specific Considerations ### 9.1 Standard: Testing Network Configurations **Do This:** Write tests to validate network configurations on Hetzner Cloud, including firewall rules, load balancer settings, and VPN connections. Use tools like "nmap" or "tcpdump" in test scripts to verify that the network is configured as expected. **Why This Matters:** Correct network configuration is essential for security and performance. ### 9.2 Standard: Testing Instance Lifecycle Events **Do This:** Implement tests to handle server lifecycle events on Hetzner Cloud, such as server creation, deletion, and resizing. **Why This Matters:** Proper handling of lifecycle events is critical for maintaining application availability and reliability. ### 9.3 Standard: Resource Cleanup **Do This:** Ensure that all test resources created on Hetzner Cloud are automatically cleaned up after each test run to avoid unnecessary costs. ## 10. Future Trends ### 10.1 Continuous Testing Embrace the concept of continuous testing by integrating testing into every stage of the software delivery lifecycle. Use tools and practices like shift-left testing to detect issues earlier and enable faster feedback loops. ### 10.2 AI-Driven Testing Explore the use of AI and machine learning to automate test case generation, execution, and analysis. Implement AI-powered tools to identify patterns and anomalies in test data, leading to more efficient and effective testing. ### 10.3 Chaos Engineering Implement chaos engineering principles to proactively identify weaknesses and vulnerabilities in the application and infrastructure. Conduct controlled experiments to simulate failures and disruptions to assess resilience and improve system recovery. By adhering to these testing standards, developers can ensure the delivery of high-quality, reliable, and secure applications on the Hetzner Cloud platform.

# Component Design Standards for Hetzner This document outlines the coding standards for component design within the Hetzner environment. Adhering to these standards promotes maintainability, reusability, performance, and security in our codebase. These standards are specifically tailored for the Hetzner ecosystem and leverage its unique features and infrastructure offerings. ## 1. Component Architecture and Principles ### 1.1. Component Definition A component is a self-contained, reusable unit of code that encapsulates specific functionality. It should have a well-defined interface, clear responsibilities, and minimal external dependencies. In the context of Hetzner, components may represent virtual machines, networking configurations, storage volumes, or application services deployed on Hetzner Cloud or dedicated servers. **Do This:** Define components with a clear purpose and limited scope. **Don't Do This:** Create monolithic components that perform multiple unrelated tasks. **Why This Matters:** Well-defined components are easier to understand, test, and maintain. They promote code reusability and reduce the risk of introducing bugs during modifications. Overly complex components lead to code tangles, increase cognitive load, and hinder scaling. ### 1.2. Abstraction and Encapsulation Components should abstract away their internal implementation details from external consumers. Encapsulation protects the integrity of the component's state and allows for modifications without affecting other parts of the system. **Do This:** Use appropriate access modifiers (e.g., private, protected) to restrict access to internal state. Define clear interfaces or APIs for interacting with the component. **Don't Do This:** Expose internal implementation details directly to external consumers. **Why This Matters:** Abstraction and encapsulation promote loose coupling, making components more independent and reusable. This allows for easier refactoring and reduces the impact of changes on other parts of the application. ### 1.3. Single Responsibility Principle (SRP) Each component should have only one reason to change. This principle ensures that components are focused and cohesive, making them easier to understand and maintain. **Do This:** Design components with a single, well-defined responsibility. If a component performs multiple tasks, consider breaking it down into smaller, more focused components. For example, a component managing Hetzner Cloud server creation should *only* handle server provisioning and not also handle database creation. **Don't Do This:** Create components that perform multiple unrelated tasks. **Why This Matters:** SRP promotes code reusability and reduces the risk of introducing bugs during modifications. It also makes components easier to test and understand. ### 1.4. Dependency Inversion Principle (DIP) High-level modules should not depend on low-level modules. Both should depend on abstractions. Abstractions should not depend on details. Details should depend on abstractions. **Do This:** Use interfaces or abstract classes to define dependencies between components. Implementing dependency injection to manage component lifecycles ensures loose coupling. Libraries like "injector" in Python, along with proper use of Hetzner's API clients (or frameworks which encapsulate them, such as Terraform providers) enable this pattern. **Don't Do This:** Create direct dependencies between high-level and low-level modules. **Why This Matters:** DIP promotes loose coupling and makes components more independent and testable. It also allows for easier swapping of implementations without affecting other parts of the system. This is crucial in Hetzner environments, where you might need to switch between different types of storage, network configurations, or even different Hetzner Cloud vs. dedicated server setups. ### 1.5. Interface Segregation Principle (ISP) Clients should not be forced to depend on methods they do not use. **Do This:** Design interfaces that are specific to the needs of the client. If an interface has too many methods, consider breaking it down into smaller, more focused interfaces. If a component interacts directly with the Hetzner API, make sure you use only the necessary API calls and avoid bloating the component with unnecessary dependencies for less frequently used features (e.g., don't include Hetzner DNS API calls in a core compute management component). **Don't Do This:** Create monolithic interfaces with methods that are not used by all clients. **Why This Matters:** ISP reduces dependencies and makes components more independent and reusable. It also reduces the risk of introducing bugs during modifications. ## 2. Component Implementation Details for Hetzner ### 2.1. Naming Conventions Use clear and consistent naming conventions for components, classes, methods, and variables. Use a naming scheme that indicates the purpose for which resources are named. **Do This:** * Use PascalCase for class names (e.g., "HetznerCloudServerManager"). * Use camelCase for method and variable names (e.g., "createServer", "serverName"). * Use a prefix or suffix to indicate the type of component. * Use descriptive names that accurately reflect the component's purpose. Example: "HetznerFirewallManager" is clear. **Don't Do This:** * Use abbreviations or acronyms that are not widely understood. * Use generic names that do not accurately reflect the component's purpose. Example: Just "Manager" is vague. * Use names that are too long or complex. **Why This Matters:** Consistent naming conventions make code easier to read and understand. They also reduce the risk of naming conflicts. ### 2.2. Error Handling Implement robust error handling to handle unexpected situations gracefully. **Do This:** * Use try-except blocks to catch exceptions. * Log errors with sufficient detail for debugging. Consider using the Hetzner Robot API (if interacting with dedicated servers) or Hetzner Cloud API for error logging. * Provide informative error messages to users. * Consider using exception chaining to preserve the original exception context. * Implement retry mechanisms for transient errors when interacting with the Hetzner API. **Don't Do This:** * Ignore exceptions. * Swallow exceptions without logging them. * Provide generic error messages that do not help users understand the problem. **Why This Matters:** Robust error handling prevents crashes and provides valuable information for debugging. It also ensures that the application behaves gracefully in unexpected situations. ### 2.3. Security Best Practices Implement security best practices to protect sensitive data and prevent vulnerabilities. **Do This:** * Use strong authentication and authorization. Handle Hetzner API tokens securely, and do *not* hardcode them into your components. Use environment variables or key management systems. * Validate input to prevent injection attacks. * Encrypt sensitive data at rest and in transit. Consider utilizing Hetzner's network features such as private networks and firewalls to enhance security. * Regularly update dependencies to patch security vulnerabilities. * Implement least privilege for components that need to access cloud or dedicated server resources. **Don't Do This:** * Store sensitive data in plain text. * Trust user input without validation. * Ignore security warnings. **Why This Matters:** Security is crucial for protecting sensitive data and preventing attacks. Following security best practices reduces the risk of vulnerabilities and helps maintain the integrity of the system. **Example (Python) - Secure API Token Handling:** """python import os import hcloud class HetznerCloudServerManager: def __init__(self): # Retrieve the HETZNER_CLOUD_API_TOKEN from the environment token = os.environ.get("HETZNER_CLOUD_API_TOKEN") if not token: raise ValueError("HETZNER_CLOUD_API_TOKEN environment variable not set.") self.client = hcloud.Client(token=token) def create_server(self, name, server_type, image): try: response = self.client.servers.create( name=name, server_type=server_type, image=image ) return response.server except hcloud.APIException as e: print(f"Error creating server: {e}") raise # Example Usage: try: server_manager = HetznerCloudServerManager() new_server = server_manager.create_server( name="my-new-server", server_type="cx11", image="ubuntu-22.04" ) print(f"Server created with ID: {new_server.id}") except ValueError as e: print(f"Configuration Error:{e}") except hcloud.APIException as e: print(f"Hetzner API Error: {e}") """ ### 2.4. Logging and Monitoring Implement comprehensive logging and monitoring to track component behavior and identify potential issues. **Do This:** * Log important events, such as component initialization, configuration changes, and errors. * Use structured logging to make it easier to analyze logs. Example: structured logging in JSON format, which can be easily ingested into monitoring tools. * Monitor component performance metrics, such as CPU usage, memory usage, and network traffic. * Set up alerts for critical events, such as errors and performance bottlenecks. Consider integration with Hetzner Monitoring. **Don't Do This:** * Log too much information (noise). * Log sensitive data (e.g., passwords, API keys). * Ignore monitoring alerts. **Why This Matters:** Logging and monitoring provide valuable insights into component behavior and help identify potential issues before they become critical. They also facilitate debugging and troubleshooting. **Example (Python) - Structured Logging:** """python import logging import json class JsonFormatter(logging.Formatter): def format(self, record): log_record = { "timestamp": self.formatTime(record, self.datefmt), "level": record.levelname, "message": record.getMessage(), "module": record.module, "funcName": record.funcName, "lineno": record.lineno } return json.dumps(log_record) # Configure the logger logger = logging.getLogger(__name__) logger.setLevel(logging.INFO) handler = logging.StreamHandler() # or a FileHandler for file-based logging # Attach the JSON formatter formatter = JsonFormatter('%Y-%m-%dT%H:%M:%S%z') handler.setFormatter(formatter) logger.addHandler(handler) # Example logging usage logger.info("Server started", extra={"server_name": "my-server", "ip_address": "192.168.1.100"}) """ ### 2.5. Configuration Management Manage component configuration in a consistent and centralized manner. **Do This:** * Use configuration files or environment variables to store configuration settings. * Use a configuration management tool to automate the deployment and management of configuration files (e.g., Ansible, Terraform). Terraform, in particular, is powerful in managing Hetzner resources. * Separate configuration from code to improve flexibility and maintainability. * Store configuration in a version control system. * Consider using a configuration server for dynamic configuration updates. **Don't Do This:** * Hardcode configuration settings in code. * Store sensitive configuration settings in plaintext. **Why This Matters:** Proper configuration management makes it easier to deploy, manage, and update components. It also reduces the risk of configuration errors. **Example (Terraform) - Hetzner Cloud Configuration** """terraform terraform { required_providers { hcloud = { source = "hetznercloud/hcloud" version = "~> 1.35.0" } } } provider "hcloud" { token = var.hcloud_token } variable "hcloud_token" { type = string sensitive = true description = "Hetzner Cloud API token" } variable "server_name" { type = string default = "web-server-01" description = "Name of the server" } resource "hcloud_server" "web" { name = var.server_name server_type = "cx11" image = "ubuntu-22.04" location = "fsn1" #Frankfurt } output "server_public_ip" { value = hcloud_server.web.ipv4_address description = "Public IP of the web server" } """ This example showcases that configuration such as the "hcloud_token" is defined in a variable and marked sensitive, ensuring it's not hardcoded directly in the Terraform code. ### 2.6. Testing Write unit tests, integration tests, and end-to-end tests to ensure component functionality. **Do This:** * Write unit tests to verify the behavior of individual components in isolation. * Write integration tests to verify the interaction between components. * Write end-to-end tests to verify the behavior of the entire system. * Use a test-driven development (TDD) approach where possible. * Automate testing as part of the build process (CI/CD). **Don't Do This:** * Skip testing. * Write tests that are too brittle or that rely on implementation details. * Ignore failing tests. **Why This Matters:** Thorough testing ensures that components function correctly and reduces the risk of introducing bugs. It also makes it easier to refactor and maintain code. **Example (Python with pytest) - Unit Testing:** """python import pytest from hcloud import Client class MockHetznerClient: def __init__(self, server_return_value=None, error=None): self.server_return_value = server_return_value self.error = error self.servers = MockHetznerServers(server_return_value, error) class MockHetznerServers: def __init__(self, server_return_value=None, error=None): self.server_return_value = server_return_value self.error = error def create(self, *args, **kwargs): if self.error: raise self.error return self.server_return_value class HetznerCloudServerManager: def __init__(self, client): self.client = client def create_server(self, name, server_type, image): try: response = self.client.servers.create( name=name, server_type=server_type, image=image ) return response.server except Exception as e: raise @pytest.fixture def mock_server(): class Server: id = 123 return Server() def test_create_server_success(mock_server): mock_client = MockHetznerClient(server_return_value=type('obj', (object,), {'server': mock_server})()) server_manager = HetznerCloudServerManager(client=mock_client) new_server = server_manager.create_server(name="test-server", server_type="cx11", image="ubuntu-22.04") assert new_server.id == 123 def test_create_server_failure(): mock_client = MockHetznerClient(error=Exception("API Error")) server_manager = HetznerCloudServerManager(client=mock_client) with pytest.raises(Exception, match="API Error"): server_manager.create_server(name="test-server", server_type="cx11", image="ubuntu-22.04") """ This example provides a basic outline of unit tests for the given component, mocking the Hetzner Client to ensure tests are isolated. ## 3. Hetzner-Specific Considerations ### 3.1 Regional Availability Hetzner offers cloud services in multiple regions. Components should be designed to be region-aware. **Do This:** Parameterize components to take region as an argument, and leverage the Hetzner API to dynamically discover available resources in a particular region. Design components to fail gracefully or switch to another available region if resources in one region are unavailable. **Don't Do This:** Hardcode region settings in components. **Why This Matters:** Regional awareness allows for greater flexibility and resilience. Hetzner's regions give options for low-latency connections to different geographical areas. ### 3.2. Network Configuration Leverage Hetzner's networking features, such as private networks and firewalls, to enhance security and performance. **Do This:** Use Hetzner's private networks to isolate sensitive components. Use Hetzner's cloud firewalls to limit access to components. Properly tag relevant firewall rules to maintain consistency. **Don't Do This:** Expose components to the public internet without proper security measures. **Why This Matters:** Proper network configuration is crucial for protecting sensitive data and preventing attacks. ### 3.3. Storage Options Hetzner offers a variety of storage options, including local storage, block storage, and object storage. Choose the appropriate storage option based on the component's requirements. **Do This:** Use local storage for temporary data. Use block storage for persistent data that requires high performance. Use object storage for large amounts of unstructured data, such as backups and media files, utilizing bucket lifecycle policies if appropriate. **Don't Do This:** Use local storage for persistent data. Misuse object storage for transactional data. **Why This Matters:** Choosing the appropriate storage option ensures optimal performance and cost-effectiveness. ### 3.4. Hetzner Load Balancers If your architecture involves load balancing, make sure that your components are designed to correctly configure and interact with Hetzner Cloud Load Balancers to distribute traffic efficiently across your VMs. The configuration should typically include proper health checks for backend VMs and sensible traffic distribution algorithms. **Do This:** Incorporate checks in your components during deploy or initialization to ensure that load balancers configurations are as expected based on declared targets. **Don't Do This:** Bypass the use of load balances for complex setups and use the cloud instances as a direct entrypoint to your apps. **Why this matters:** Load balancers are managed at the infrastructure level by Hetzner, thus making them an optimal choice to ensure availability with managed health checks. By adhering to these coding standards, we can ensure that our components are well-designed, maintainable, reusable, and secure. This will ultimately contribute to the success of our projects deployed within the Hetzner environment. This document should be continuously improved with new best practices and lessons learned.

# Performance Optimization Standards for Hetzner This document outlines performance optimization standards for developing applications and infrastructure on Hetzner Cloud and dedicated servers. These standards are designed to improve application speed, responsiveness, and resource usage, taking into account Hetzner's specific environment and tools. ## 1. Infrastructure and Architecture ### 1.1 Choosing the Right Instance Type **Standard:** Select an instance type (CPU, RAM, storage) that appropriately matches your application's workload. Continuously monitor resource usage and scale accordingly. **Why:** Over-provisioning wastes resources and increases costs. Under-provisioning leads to performance bottlenecks and poor user experience. **Do This:** * Use "htop", "vmstat", "iostat", "netstat" (or their modern alternatives like "bpytop", "nethogs", "iotop") on your Hetzner server to monitor CPU, memory, disk I/O, and network usage. * Utilize Hetzner Cloud's metrics API or Cloud Console to track resource consumption over time. * Conduct load testing to simulate real-world traffic and identify performance bottlenecks. * Compare different instance types using benchmark tools like "sysbench" or "fio". **Don't Do This:** * Assume that a larger instance is always better. Optimize your application first. * Rely solely on anecdotal evidence or gut feelings when choosing an instance type. * Ignore resource usage metrics after the initial deployment. **Code Example (Bash - Monitoring CPU Usage):** """bash #!/bin/bash # Get CPU usage as a percentage cpu_usage=$(top -bn1 | grep "Cpu(s)" | sed "s/.*, *\([0-9.]*\)%* id.*/\1/" | awk '{print 100 - $1}') echo "Current CPU Usage: $cpu_usage%" # Example threshold - send an alert if CPU exceeds 80% if (( $(echo "$cpu_usage > 80" | bc -l) )); then echo "WARNING: CPU usage is high!" # Add your alert logic here (e.g., send an email) fi """ ### 1.2 Leveraging Hetzner Load Balancers **Standard:** Use Hetzner Load Balancers to distribute traffic across multiple servers, improving availability and performance under high load. **Why:** Load balancers prevent single points of failure and distribute workload, ensuring consistent performance even during traffic spikes. **Do This:** * Configure health checks to ensure that only healthy servers receive traffic. * Use the appropriate load balancing algorithm (e.g., round robin, least connections) based on your application's needs. * Enable session persistence (sticky sessions) if required by your application. * Monitor load balancer metrics to identify potential issues. **Don't Do This:** * Rely on a single server to handle all traffic. * Forget to configure health checks. * Ignore load balancer metrics. **Code Example (Hetzner Cloud API - Creating a Load Balancer):** """python import hcloud import os # Replace with your Hetzner Cloud API token HCLOUD_TOKEN = os.environ["HCLOUD_TOKEN"] # Get the API client client = hcloud.Client(token=HCLOUD_TOKEN) name = "my-load-balancer" load_balancer_type = client.load_balancer_type.get_by_name("lb11") #Choose instance type datacenter = client.datacenters.get_by_name("hel1-dc2") #choose datacenter response = client.load_balancer.create( name=name, load_balancer_type=load_balancer_type, location=datacenter # This can also be a location e.g. 'fsn1' for Falkenstein ) load_balancer = response.load_balancer print(f"Load Balancer created with ID: {load_balancer.id}") """ ### 1.3 Using a Content Delivery Network (CDN) **Standard:** Utilize a CDN like Cloudflare or BunnyCDN (or others) to serve static assets (images, CSS, JavaScript) closer to your users, reducing latency and improving page load times. **Why:** CDNs cache content in multiple locations around the world, improving delivery speed for geographically dispersed users served by Hetzner. **Do This:** * Configure your CDN to cache static assets with appropriate cache control headers. * Use a CDN URL for all static assets in your application. * Purge the CDN cache when you update static assets. **Don't Do This:** * Serve static assets directly from your Hetzner server’s local filesystem for global audience. * Forget to configure cache control headers. * Oversize assets that impact CDN performance. ### 1.4 Database Optimization **Standard:** Optimize database queries, schema design, and caching to reduce database load and improve application performance. **Why:** Databases are often a bottleneck in web applications. Efficient database operations are crucial for performance. **Do This:** * Use indexes to speed up queries. Analyze slow queries using tools like "EXPLAIN" (MySQL/MariaDB) or "EXPLAIN ANALYZE" (PostgreSQL). * Optimize your database schema to reduce redundancy and improve data integrity. * Utilise connection pooling. * Implement caching mechanisms (e.g., Memcached, Redis) to reduce database load. * Consider using a read replica for read-heavy operations. Hetzner doesn't provide managed database services, so you'll need to set these up manually or use a managed service from another provider. **Don't Do This:** * Perform full table scans unnecessarily. * Store large binary data (images, videos) directly in the database, instead use Object Storage and store object storage URLs reference in the database. * Ignore slow query logs or database performance monitoring tools. **Code Example (PostgreSQL - Indexing and Query Optimization):** """sql -- Create an index on the 'email' column CREATE INDEX idx_users_email ON users (email); -- Analyze a slow query EXPLAIN ANALYZE SELECT * FROM orders WHERE user_id = 123 AND order_date BETWEEN '2023-01-01' AND '2023-12-31'; -- Rewrite the query to use the index more efficiently if the explain plan indicates it's not being used effectively. """ ## 2. Application-Level Optimization ### 2.1 Code Profiling and Optimization **Standard:** Use profiling tools to identify performance bottlenecks in your code and optimize accordingly. **Why:** Profiling reveals the areas of your code that consume the most resources, allowing you to focus your optimization efforts effectively. **Do This:** * Use profiling tools like "cProfile" (Python), Xdebug (PHP), or built-in profilers in your chosen language/framework. * Identify slow functions and analyze their execution time. * Optimize algorithms and data structures for better performance. **Don't Do This:** * Guess where performance bottlenecks are. Use profiling to find them. * Optimize prematurely without profiling. * Ignore warnings and errors reported by profiling tools. **Code Example (Python - Using cProfile):** """python import cProfile import pstats def my_slow_function(): """A function that takes a long time to execute (for demonstration purposes).""" result = 0 for _ in range(1000000): result += 1 return result # Profile the function cProfile.run('my_slow_function()', 'profile_output') # Analyze the profiling results p = pstats.Stats('profile_output') p.sort_stats('cumulative').print_stats(10) # Show the 10 most time-consuming functions """ ### 2.2 Caching Strategies **Standard:** Implement caching at various levels (browser, server-side, database) to reduce response times and improve scalability. **Why:** Caching prevents repetitive computations and data retrieval, significantly improving performance. **Do This:** * Use browser caching for static assets (images, CSS, JavaScript) by setting appropriate HTTP headers (e.g., "Cache-Control", "Expires"). * Implement server-side caching using tools like Memcached or Redis. * Cache frequently accessed database query results. * Implement HTTP caching with reverse proxies like Varnish, particularly effective for serving static pages/content. **Don't Do This:** * Cache sensitive data without proper security measures. * Cache data indefinitely without a mechanism for invalidation. * Over-cache, leading to stale data. **Code Example (Python - Using Redis for Server-Side Caching):** """python import redis import time # Connect to Redis (replace with your Hetzner server details if Redis is not on localhost) redis_client = redis.Redis(host='localhost', port=6379, db=0) def get_data_from_expensive_source(key): """Simulates fetching data from a slow data source (e.g., database query, API call).""" time.sleep(2) # Simulate a slow process return f"Data for {key} from expensive source" def get_data(key): """Retrieves data from the cache or the expensive source.""" cached_data = redis_client.get(key) if cached_data: print("Serving from cache") return cached_data.decode('utf-8') # Decode bytes to string else: print("Fetching from expensive source") data = get_data_from_expensive_source(key) redis_client.set(key, data, ex=3600) # Cache for 1 hour (3600 seconds) return data # Example usage print(get_data("my_data")) # First call: Fetches from the expensive source and caches it print(get_data("my_data")) # Second call: Serves from the cache """ ### 2.3 Asynchronous Operations **Standard:** Use asynchronous operations for long-running tasks (e.g., sending emails, processing images) to prevent blocking the main thread and improve responsiveness. **Why:** Asynchronous operations allow your application to handle multiple requests concurrently, improving overall throughput and responsiveness. **Do This:** * Use asynchronous task queues like Celery or RabbitMQ. * Offload long-running tasks to background workers. * Use asynchronous I/O libraries (e.g., "asyncio" in Python) for network operations. * Utilize serverless functions on the Hetzner infrastructure on demand. **Don't Do This:** * Perform long-running tasks in the main request thread. * Block the event loop with synchronous operations. **Code Example (Python - Using Celery for Asynchronous Tasks):** """python from celery import Celery import time # Celery configuration (replace with your RabbitMQ/Redis details) celery_app = Celery('my_tasks', broker='redis://localhost:6379/0', backend='redis://localhost:6379/0') @celery_app.task def send_email(email_address, message): """Sends an email asynchronously (simulated).""" print(f"Sending email to {email_address}...") time.sleep(5) # Simulate sending an email (takes time) print(f"Email sent to {email_address}: {message}") return True # Example usage (in your web application): # send_email.delay('user@example.com', 'Welcome to our service!') """ To run the above Celery example, install Celery and Redis: "pip install celery redis". Start the Celery worker: "celery -A your_module worker -l info". Replace "your_module" with the name of the file containing the Celery configuration and task definitions. Then, you can call the "send_email.delay(...)" function from your web application. ### 2.4 Minimizing Network Latency **Standard:** Reduce network latency by choosing server locations closer to your users and optimizing network communication. **Why:** Network latency can significantly impact application performance, especially for geographically dispersed users. **Do This:** * Choose Hetzner datacenters closer to your target audience. Hetzner offers datacenters in multiple locations. * Use HTTP/2 for improved network efficiency. * Enable compression (e.g., Gzip, Brotli) for text-based assets. * Minimize the number of HTTP requests by bundling CSS and JavaScript files. **Don't Do This:** * Ignore the geographic location of your users. * Use inefficient network protocols. * Send uncompressed data over the network. ### 2.5 Image Optimization **Standard:** Optimize images by compressing them, resizing them to appropriate dimensions, and using modern image formats (e.g., WebP, AVIF). **Why:** Large, unoptimized images can significantly slow down page load times and waste bandwidth. **Do This:** * Use image optimization tools like "optipng", "jpegoptim", or "webp". * Resize images to the dimensions required by your application. * Use responsive images with the "<picture>" element to serve different image sizes based on screen size. * Consider using a CDN that automatically optimizes images. * Favor modern image formats like WebP or AVIF over older formats like JPEG and PNG where browser support permits. **Don't Do This:** * Upload large, unoptimized images directly to your server. * Use the same image size for all screen sizes. * Ignore the potential of modern image codecs. **Code Example (Bash - Using "cwebp" to convert JPEG to WebP):** """bash #!/bin/bash # Convert a JPEG image to WebP cwebp -q 80 input.jpg -o output.webp echo "Image converted to WebP" """ ## 3. Hetzner-Specific Optimizations ### 3.1 Hetzner Cloud Object Storage **Standard:** Use Hetzner Cloud Object Storage for storing large files (images, videos, documents) to reduce load on your servers and improve scalability. **Why:** Object Storage is a cost-effective and scalable solution for storing large amounts of data. **Do This:** * Store static assets in Object Storage. * Configure appropriate access control policies. * Use a CDN in front of Object Storage for improved delivery speed. **Don't Do This:** * Store temporary files or frequently accessed data in Object Storage. * Grant public access to sensitive data. ### 3.2 IPv6 Optimization **Standard:** Configure your servers and applications to use IPv6 for improved network performance and future compatibility. **Why:** IPv6 offers several advantages over IPv4, including larger address space and improved routing efficiency. Hetzner fully supports IPv6. **Do This:** * Enable IPv6 on your Hetzner Cloud servers. * Configure your DNS records to include IPv6 addresses. * Test your application's IPv6 compatibility. **Don't Do This:** * Ignore IPv6. * Assume that IPv4 is sufficient for all your needs. ### 3.3 Hetzner Robot Management **Standard:** Automate server management tasks using the Hetzner Robot API for efficient scaling and configuration. **Why:** Automating tasks reduces manual effort and ensures consistent server configuration. **Do This:** * Use the Hetzner Robot API to create, configure, and manage servers. * Use configuration management tools like Ansible or Terraform to automate server deployments. * Monitor server hardware health using the Robot API. **Don't Do This:** * Perform server management tasks manually. * Ignore server hardware alerts. ## 4. Monitoring and Alerting **Standard:** Implement comprehensive monitoring and alerting to identify performance issues proactively. **Why:** Early detection of performance issues allows you to address them before they impact users. **Do This:** * Use monitoring tools like Prometheus, Grafana, or Nagios to track server and application metrics. * Set up alerts for critical metrics (CPU usage, memory usage, disk I/O, network traffic, response times). * Implement log aggregation and analysis using tools like the ELK stack (Elasticsearch, Logstash, Kibana) or Splunk. * Regularly review monitoring data and adjust your configuration as needed. **Don't Do This:** * Ignore monitoring data. * Set excessively high or low alert thresholds or overly sensitive alerts. * Fail to investigate alerts promptly. ## 5. Code Style and Readability **Standard:** Write clean, well-documented, and maintainable code. **Why:** Readability increases operational efficiency and reduces errors. **Do This:** * Follow consistent code style guidelines (e.g., PEP 8 for Python, PSR for PHP). * Use meaningful variable and function names. * Write clear and concise comments. * Break down complex functions into smaller, more manageable units. * Use version control (Git) and follow a consistent branching strategy. **Don't Do This:** * Write overly complex or convoluted code. * Ignore code style guidelines. * Fail to document your code adequately. By adhering to these performance optimization standards, you can build high-performing, scalable, and reliable applications on Hetzner Cloud and dedicated servers. Regularly review and update these standards to keep pace with evolving technologies and best practices.

# Deployment and DevOps Standards for Hetzner This document outlines the deployment and DevOps standards for software development projects targeting Hetzner Cloud and related Hetzner services. It is intended to guide developers in creating robust, scalable, and maintainable applications within the Hetzner ecosystem. These standards will be used as context for AI coding assistants. ## 1. Infrastructure as Code (IaC) & Configuration Management By treating infrastructure as code, we enable version control, automated deployments, and repeatable environments. **Standard:** Use Infrastructure as Code (IaC) tools for managing Hetzner Cloud resources. * **Do This:** Use Terraform, Pulumi, or similar IaC tools to define and manage Hetzner Cloud resources (servers, networks, firewalls, etc.). * **Don't Do This:** Manually create and configure resources through the Hetzner Cloud Console. Manual changes lead to configuration drift and make infrastructure harder to reproduce and manage. **Why:** IaC provides version control, auditability, and repeatability for infrastructure provisioning. It enables automated deployments and facilitates disaster recovery. Hetzner's API is well-suited for programmatic management, maximizing the benefits of IaC. **Code Example (Terraform):** """terraform # Configure the Hetzner Cloud Provider terraform { required_providers { hcloud = { source = "hetznercloud/hcloud" version = "~> 1.35.0" # Use the latest stable version } } } provider "hcloud" { token = var.hcloud_token } # Create a Hetzner Cloud Server resource "hcloud_server" "main" { name = "web-server-01" server_type = "cx11" # Change to an appropriate server type image = "ubuntu-22.04" # Use a supported and current image. Check Hetzner's supported images. location = "fsn1" # Choose appropriate location for your needs user_data = file("cloud-init.yaml") # Cloud-init for initial server configuration network { network_id = hcloud_network.my_network.id ip = "10.0.1.10" } depends_on=[hcloud_network.my_network, hcloud_firewall.allow_http] } resource "hcloud_network" "my_network" { name = "my-private-network" ip_range = "10.0.0.0/16" location = "fsn1" } resource "hcloud_firewall" "allow_http" { name = "allow-http" rule { direction = "in" protocol = "tcp" port = "80" source_ips = ["0.0.0.0/0", "::/0"] } rule { direction = "in" protocol = "tcp" port = "443" source_ips = ["0.0.0.0/0", "::/0"] } apply_to { type = "server" server = hcloud_server.main.id } depends_on=[hcloud_network.my_network] } output "server_public_ip" { value = hcloud_server.main.ipv4_address } """ **Anti-Pattern:** Hardcoding sensitive information (API tokens, passwords) directly in IaC configuration files. Always use environment variables, secrets managers, or Terraform's input variables with appropriate sensitivity settings. ### 1.1 Configuration Management Tooling Beyond infrastructure provisioning, use configuration management tools to ensure consistent application deployment and configuration across servers. **Standard:** Use Ansible, Chef, Puppet, or SaltStack to automate application deployment and configuration. * **Do This:** Define application dependencies, configurations, and deployment steps in configuration management playbooks/recipes/modules. * **Don't Do This:** Manually configure applications on each server. Manual configuration opens the door for inconsistencies and requires significant manual labor for updates and rollbacks. **Why:** Configuration management ensures consistency and reduces the risk of human error during deployments. It streamlines application updates and rollbacks and enables automated infrastructure scaling. **Code Example (Ansible):** """yaml --- - hosts: web_servers become: true tasks: - name: Update apt cache apt: update_cache: yes become: yes - name: Install Nginx apt: name: nginx state: present become: yes - name: Copy Nginx configuration file copy: src: nginx.conf dest: /etc/nginx/nginx.conf become: yes notify: Restart Nginx - name: Enable Nginx site file: src: /etc/nginx/sites-available/default dest: /etc/nginx/sites-enabled/default state: absent become: yes notify: Restart Nginx handlers: - name: Restart Nginx service: name: nginx state: restarted become: yes """ **Anti-Pattern:** Storing secrets directly in configuration management playbooks. Use Ansible Vault or similar mechanisms to encrypt sensitive data. ### 1.2 Cloud-init for Initial Server Setup Leverage "cloud-init" to automate the initial configuration of Hetzner Cloud servers during provisioning. **Standard:** Use "cloud-init" for tasks such as setting the hostname, installing packages, and configuring users upon server creation. * **Do This:** Provide a "cloud-init" configuration file when creating Hetzner Cloud servers via IaC or the API. * **Don't Do This:** Rely solely on manual configuration after a server is provisioned. This adds to manual effort and slows down the initial setup. **Why:** "cloud-init" automates the bootstrapping process, saving time and ensuring consistency across servers. It integrates seamlessly with Hetzner Cloud's server creation process. **Code Example (Cloud-init):** """yaml #cloud-config hostname: my-webserver packages: - nginx - git users: - name: deploy groups: sudo shell: /bin/bash ssh_authorized_keys: - ssh-rsa AAAAB3NzaC deploy@example.com write_files: - path: /var/www/html/index.html owner: www-data:www-data permissions: '0644' content: | <h1>Hello, World!</h1> """ **Anti-Pattern:** Embedding sensitive information in "cloud-init" configurations. Consider using "cloud-init" to retrieve secrets from a secure store after the instance is initialized. ## 2. Continuous Integration and Continuous Delivery (CI/CD) Automate the build, test, and deployment process to ensure rapid and reliable software releases to Hetzner Cloud. **Standard:** Implement a CI/CD pipeline using tools like Jenkins, GitLab CI, GitHub Actions, or CircleCI. * **Do This:** Create automated pipelines that build, test, and deploy code changes to staging and production environments on Hetzner Cloud. * **Don't Do This:** Manually build and deploy applications. Manual deployments are error-prone, slow, and hinder rapid iteration. **Why:** CI/CD automates the software release process, reducing the risk of errors and enabling faster delivery cycles. It promotes collaboration and improves code quality. **Code Example (GitHub Actions):** """yaml name: Deploy to Hetzner Cloud on: push: branches: - main jobs: deploy: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Install dependencies run: npm install # Or equivalent for your language - name: Build application run: npm run build # Or equivalent - name: Deploy to Hetzner Cloud uses: appleboy/scp-action@v0.1.4 with: host: ${{ secrets.HETZNER_SERVER_IP }} # Store IP as secret in github actions settings username: deploy key: ${{ secrets.HETZNER_SSH_KEY }} # Store server SSH key in github actions settings port: 22 source: build # The output directory from your build process target: /var/www/html - name: SSH remote command uses: appleboy/ssh-action@v0.1.10 with: host: ${{ secrets.HETZNER_SERVER_IP }} username: deploy key: ${{ secrets.HETZNER_SSH_KEY }} port: 22 script: | sudo systemctl restart nginx """ **Anti-Pattern:** Lack of automated testing in the CI/CD pipeline. Ensure that unit tests, integration tests, and end-to-end tests are run automatically to catch errors early. ### 2.1 Deployment Strategies Choose appropriate deployment strategies based on application requirements and risk tolerance. **Standard:** Implement strategies such as blue/green deployments, rolling updates, or canary releases in production. * **Do This:** Use blue/green deployments for zero-downtime releases. Rolling updates are suitable for less critical applications. Canary releases allow testing new code with a small subset of users. * **Don't Do This:** Deploy directly to production without a proper deployment strategy that minimizes downtime and risk. **Why:** Deployment strategies minimize disruptions during deployments and enable easy rollback if issues arise. **Example (Blue/Green Deployment – Conceptual):** 1. Provision two identical environments: blue (current version) and green (new version). 2. Deploy the new version to the green environment. 3. Run tests against the green environment. 4. Switch the load balancer to direct traffic to the green environment, making it live. The blue environment becomes the backup. 5. If issues arise, quickly switch the load balancer back to the blue environment. Implementing this effectively requires IaC and automation for provisioning and environment switching. ### 2.2 Containerization (Docker) Use containerization to package applications with all their dependencies, ensuring consistency across different environments. **Standard:** Containerize applications using Docker and use Docker Compose for multi-container applications. * **Do This:** Create Dockerfiles that define the application's dependencies and build process. Use Docker Compose to orchestrate multi-container deployments on Hetzner Cloud. * **Don't Do This:** Deploy applications directly without containerization. This increases complexity and may lead to environment config differences that cause runtime issues. **Why:** Containerization provides portability, isolation, and reproducibility for applications. It simplifies deployments and ensures consistency across different environments. **Code Example (Dockerfile):** """dockerfile FROM ubuntu:22.04 RUN apt-get update && apt-get install -y nginx COPY nginx.conf /etc/nginx/nginx.conf COPY index.html /var/www/html/index.html EXPOSE 80 CMD ["nginx", "-g", "daemon off;"] """ **Code Example (Docker Compose):** """yaml version: "3.9" services: web: image: my-nginx-image:latest ports: - "80:80" restart: always # Automatically restart the container if it fails networks: - webnet database: image: postgres:latest environment: POSTGRES_USER: myuser POSTGRES_PASSWORD: mypassword volumes: - db_data:/var/lib/postgresql/data networks: - webnet networks: webnet: volumes: db_data: """ **Anti-Pattern:** Storing sensitive data within the Docker image. Utilize Docker secrets or environment variables for passing sensitive information to containers. ### 2.3 Hetzner Container Registry Leverage the Hetzner Container Registry for storing and managing Docker images. **Standard:** Use the Hetzner Container Registry for secure and efficient storage of Docker images used in deployments on Hetzner Cloud. * **Do This:** Push Docker images to the Hetzner Container Registry from your CI/CD pipeline. * **Don't Do This:** Rely on public Docker Hub for private or sensitive images. **Why:** The Hetzner Container Registry provides a secure, private, and performant solution for storing Docker images within the Hetzner ecosystem and avoids egress charges by keeping the images within the Hetzner network. **Example (Pushing to Hetzner Container Registry):** """bash docker tag my-nginx-image:latest registry.your-hetzner-registry.com/your-project/my-nginx-image:latest docker login -u your-username -p your-password registry.your-hetzner-registry.com docker push registry.your-hetzner-registry.com/your-project/my-nginx-image:latest """ ## 3. Monitoring and Logging Implement comprehensive monitoring and logging to gain insights into application performance and detect issues proactively. **Standard:** Use a centralized logging system and a monitoring solution to track application and infrastructure health. * **Do This:** Implement logging to a central location using tools like Fluentd, Logstash, or Graylog. Use Prometheus and Grafana for monitoring key metrics (CPU, memory, disk usage, network I/O, application response times). * **Don't Do This:** Rely on local logs on servers or lack real-time monitoring of key performance indicators. **Why:** Monitoring and logging provide visibility into application behavior, enabling proactive identification of performance bottlenecks, errors, and security threats. Centralized logging simplifies troubleshooting and analysis. **Code Example (Prometheus Exporter – Conceptual):** This assumes you're exposing metrics from your application. Many languages have readily available Prometheus client libraries. """python # Example Python code using the Prometheus client library from prometheus_client import start_http_server, Summary import random import time # Create a metric to track time spent and requests made. REQUEST_TIME = Summary('request_processing_seconds', 'Time spent processing request') # Decorate function with metric. @REQUEST_TIME.time() def process_request(t): """A dummy function that takes some time.""" time.sleep(t) if __name__ == '__main__': # Start up the server to expose the metrics. start_http_server(8000) # Generate some requests. while True: process_request(random.random()) """ You would then configure Prometheus to scrape this endpoint to collect the metrics. ### 3.1 Hetzner Cloud Metrics Utilize the Hetzner Cloud API to gather metrics about your servers and network infrastructure. **Standard:** Collect and monitor key metrics exposed by the Hetzner Cloud API, such as CPU usage, memory usage, disk I/O, and network traffic. * **Do This:** Integrate the Hetzner Cloud API with your monitoring system (e.g., Prometheus) to gather resource utilization data. * **Don't Do This:** Ignore the metrics provided by the Hetzner Cloud API, leading to incomplete visibility into infrastructure performance. **Why:** Hetzner Cloud metrics provide valuable insights into the performance and health of your infrastructure, enabling you to identify and address issues proactively. ### 3.2 Alerting Configure alerts to notify operators when critical events occur, such as high CPU usage, low disk space, or application errors. **Standard:** Set up alerts in your monitoring system to trigger notifications when predefined thresholds are exceeded or when specific events occur. * **Do This:** Configure alerts based on SLOs and business-critical metrics. Use alerting tools like Alertmanager (with Prometheus) or similar functionalities in other monitoring platforms. * **Don't Do This:** Fail to configure alerts, leading to delayed detection of critical issues and potential downtime. **Why:** Alerting enables rapid response to critical events, minimizing downtime and preventing performance degradation. ## 4. Security Best Practices Implement security measures at all levels of the deployment pipeline, from infrastructure provisioning to application deployment. **Standard:** Follow security best practices for infrastructure, application, and data protection. * **Do This:** Use strong passwords, enable multi-factor authentication, regularly update software, implement firewalls, and encrypt data at rest and in transit. Refer to Hetzner's security recommendations and compliance documentation. * **Don't Do This:** Neglect security considerations, leading to vulnerabilities and potential data breaches. **Why:** Security best practices protect against unauthorized access, data loss, and service disruptions. ### 4.1 Firewall Management Use Hetzner Cloud Firewalls to control network traffic to and from servers. **Standard:** Implement Hetzner Cloud Firewalls to restrict access to servers and services based on the principle of least privilege. * **Do This:** Create firewall rules that allow only necessary traffic to specific ports and IP addresses. Use separate firewalls for different environments (e.g., staging, production). * **Don't Do This:** Leave firewalls disabled or open up all ports to all traffic. **Why:** Firewalls prevent unauthorized access to servers and services, reducing the risk of security breaches. **Code Example (Terraform - Firewall Revisited from above):** Re-emphasizing the firewall example to show its importance. """terraform resource "hcloud_firewall" "allow_http" { name = "allow-http" rule { direction = "in" protocol = "tcp" port = "80" source_ips = ["0.0.0.0/0", "::/0"] } rule { direction = "in" protocol = "tcp" port = "443" source_ips = ["0.0.0.0/0", "::/0"] } rule { direction = "in" protocol = "tcp" port = "22" # Restrict access to SSH only to your IP address source_ips = ["YOUR_IP_ADDRESS/32", "YOUR_IPV6_ADDRESS/128"] # Replace with your actual IP addresses } apply_to { type = "server" server = hcloud_server.main.id } depends_on=[hcloud_network.my_network] } """ **Anti-Pattern:** Using a broad "0.0.0.0/0" IP range and leaving port 22 (SSH) open to the world. ### 4.2 Secrets Management Store sensitive information (API keys, passwords, certificates) securely and prevent them from being exposed in code or configuration files. **Standard:** Use a secrets management solution like HashiCorp Vault, AWS Secrets Manager (if using AWS in conjunction with Hetzner), or the Hetzner Key Management Service (KMS). * **Do This:** Store secrets in a secrets management system and inject them into applications at runtime. Rotate secrets regularly to minimize the impact of potential compromises. * **Don't Do This:** Hardcode secrets in code, configuration files, or environment variables without proper encryption. **Why:** Secrets management protects sensitive information from unauthorized access, reducing the risk of data breaches. ## 5. Scalability and High Availability Design applications and infrastructure for scalability and high availability to ensure continuous operation and handle increasing traffic. **Standard:** Design applications to be stateless and horizontally scalable. Use load balancers to distribute traffic across multiple servers. Implement redundancy to eliminate single points of failure. * **Do This:** Use Hetzner Load Balancers to distribute traffic across multiple application servers. Implement database replication and clustering for high availability. Use object storage (e.g., Hetzner Storage Box) for scalable storage of static assets. * **Don't Do This:** Rely on a single server for critical applications or store session state locally. This creates a single point of failure and limits scalability. **Why:** Scalability and high availability ensure continuous operation, even during peak traffic or server failures. ## 6. Backup and Disaster Recovery Establish a backup and disaster recovery plan to protect against data loss and ensure business continuity. **Standard:** Implement regular backups of data and system configurations. Create a disaster recovery plan that outlines the steps to restore services in the event of a major outage. * **Do This:** Use Hetzner Cloud snapshots for regular backups of servers. Test the disaster recovery plan periodically to ensure it is effective. Consider using Hetzner Storage Box for offsite backups. * **Don't Do This:** Neglect backups and disaster recovery planning, leading to potential data loss and prolonged downtime in the event of a disaster. **Why:** A backup and disaster recovery plan protects against data loss and ensures business continuity in the event of a major outage. These standards provide a comprehensive guide for deployment and DevOps best practices within the Hetzner Cloud environment. Adhering to these guidelines will enable developers to create robust, scalable, secure, and maintainable applications.

# Tooling and Ecosystem Standards for Hetzner This document outlines the recommended tooling and ecosystem standards for developing applications and infrastructure relating to Hetzner Cloud. Adhering to these guidelines will promote consistency, maintainability, performance, and security across all Hetzner-related projects. It's designed to be used by developers and as context for AI coding assistants to generate code that adheres to these best practices. ## 1. Infrastructure as Code (IaC) ### Standards * **Do This:** Embrace Infrastructure as Code (IaC) using tools like Terraform, Ansible, or Pulumi to manage Hetzner Cloud resources programmatically and reproducibly. * **Don't Do This:** Manually provision resources through the Hetzner Cloud Console except for initial exploration and prototyping. Manual changes create configuration drift and hinder reproducibility. ### Why IaC Matters * **Reproducibility:** IaC enables the creation of identical environments, minimizing inconsistencies between development, testing, and production. * **Version Control:** IaC configurations reside in version control systems (e.g., Git), allowing for tracking changes, collaboration, and easy rollbacks. * **Automation:** Automates infrastructure provisioning and management, reducing manual effort and the risk of errors. * **Auditability:** Provides a clear audit trail of infrastructure changes. ### Terraform Example """terraform # Configure the Hetzner Cloud Provider terraform { required_providers { hcloud = { source = "hetznercloud/hcloud" version = "~> 1.42.0" # Latest version at time of writing } } } provider "hcloud" { token = var.hcloud_token } # Create a Hetzner Cloud Server resource "hcloud_server" "main" { name = "web-server-01" server_type = "cx11" image = "ubuntu-22.04" location = "fsn1" #Falkenstein, Germany ssh_keys = [hcloud_ssh_key.default.id] user_data = file("${path.module}/cloud-init.yaml") #See below for example network { network_id = hcloud_network.private_network.id ip = "10.0.1.10" } } resource "hcloud_ssh_key" "default" { name = "my-ssh-key" public_key = file("~/.ssh/id_rsa.pub") # Replace with your actual public key } resource "hcloud_network" "private_network" { name = "my-private-network" ip_range = "10.0.0.0/16" labels = { environment = "production" } } """ Cloud-init example ("cloud-init.yaml"): """yaml #cloud-config package_update: true package_upgrade: true packages: - nginx write_files: - path: /var/www/html/index.html content: | <h1>Welcome to my Hetzner Cloud Server!</h1> runcmd: - systemctl start nginx """ **Explanation:** This example uses Terraform to define a Hetzner Cloud server, an SSH key, and a private network. It retrieves the SSH public key from a file, specifies the server type, image, and location. It also uses "cloud-init" to install Nginx and create a default web page during server creation. Using a private network helps isolate your resources. **Anti-patterns:** * Hardcoding the Hetzner Cloud API Token directly in the Terraform configuration. Always use environment variables or a secure vault. * Ignoring Terraform state management. Properly store and version your Terraform state to avoid inconsistencies. Use Terraform Cloud or a remote backend (e.g., AWS S3 with DynamoDB for locking). ### Ansible Example """yaml --- - hosts: localhost connection: local gather_facts: false tasks: - name: Create a Hetzner Cloud Server hcloud_server: name: web-server-02 server_type: cx11 image: ubuntu-22.04 location: fsn1 ssh_keys: - "{{ hcloud_ssh_key_id }}" # Define hcloud_ssh_key_id in vars or inventory api_token: "{{ hcloud_api_token }}" # Define hcloud_api_token in vars or inventory state: present register: server_result - name: Output Server Details debug: var: server_result """ **Explanation:** This Ansible playbook creates a Hetzner Cloud server. It defines the server’s name, type, image, location, and SSH key. The "hcloud_api_token" and "hcloud_ssh_key_id" should be defined as variables (ideally securely using Ansible Vault). **Anti-patterns:** * Storing the Hetzner Cloud API token directly in the Ansible playbook. Use Ansible Vault or environment variables to protect secrets. * Not using Ansible's idempotency features. Ensure your playbooks only make changes when necessary. ## 2. Hetzner Cloud API Clients and SDKs ### Standards * **Do This:** Utilize official or well-maintained community-driven Hetzner Cloud API clients/SDKs for interacting with the Hetzner Cloud API. Prefer the official SDKs whenever possible. * **Don't Do This:** Implement custom HTTP requests to the Hetzner Cloud API unless absolutely necessary. This increases the likelihood of errors and reduces code maintainability. ### Why Use API Clients/SDKs * **Abstraction:** SDKs abstract away the complexities of the Hetzner Cloud API, providing a higher-level interface for common operations. * **Error Handling:** SDKs typically include robust error handling and retry mechanisms. * **Data Serialization/Deserialization:** SDKs handle the serialization and deserialization of data between your application and the Hetzner Cloud API. * **Authentication:** SDKs simplify the authentication process with the Hetzner Cloud API. * **Updates:** SDKs are usually updated when new API features are supported. ### Python Example (using the official "hcloud" SDK) """python from hcloud import Client import os hcloud_token = os.environ.get("HCLOUD_TOKEN") if not hcloud_token: raise ValueError("HCLOUD_TOKEN environment variable must be set.") client = Client(token=hcloud_token) # Create a new server try: response = client.servers.create( name="my-python-server", server_type="cx11", image="ubuntu-22.04", location="fsn1", ) server = response.server print(f"Server created with ID: {server.id}") # Get Server details server = client.servers.get_by_name("my-python-server") if server: print(f"Server Public IP: {server.public_net.ipv4.ip}") else: print("Server not found") except Exception as e: print(f"Error creating server: {e}") """ **Explanation:** This Python code snippet uses the "hcloud" library to create a new Hetzner Cloud server. It uses "client.servers.create" to create the server instance. The code handles potential errors during server creation. It also uses "client.servers.get_by_name" retrieve a server and extract public IP address. The API token is retrieved from an environment variable. **Anti-patterns:** * Storing the Hetzner Cloud API token directly in the script. Use environment variables or a dedicated secrets management solution. * Not handling exceptions properly. Implement robust error handling to catch and log API errors. * Polling for server creation completion excessively and without backoff. Use event-driven approaches where possible or implement exponential backoff if polling is necessary. ### Go Example (using the official "hcloud-go" SDK) """go package main import ( "context" "fmt" "os" "github.com/hetznercloud/hcloud-go/hcloud" ) func main() { hcloudToken := os.Getenv("HCLOUD_TOKEN") if hcloudToken == "" { fmt.Println("HCLOUD_TOKEN environment variable must be set.") return } client := hcloud.NewClient(hcloud.WithToken(hcloudToken)) ctx := context.Background() // Create a server opts := hcloud.ServerCreateOpts{ Name: "my-go-server", ServerType: hcloud.ServerType{Name: "cx11"}, Image: hcloud.Image{Name: "ubuntu-22.04"}, Location: hcloud.Location{Name: "fsn1"}, } result, _, err := client.Server.Create(ctx, opts) if err != nil { fmt.Printf("Error creating server: %s\n", err) return } fmt.Printf("Server created with ID: %d\n", result.Server.ID) // Get Server Details server, _, err := client.Server.GetByName(ctx, "my-go-server") if err != nil { fmt.Printf("Error getting server: %s\n", err) return } if server != nil { fmt.Printf("Server Public IP: %s\n", server.PublicNet.IPv4.IP.String()) } else { fmt.Println("Server not found") } } """ **Explanation:** This Go code creates a Hetzner Cloud server named "my-go-server". It retrieves the HCLOUD_TOKEN from the environment variables. The example also retrieves the server using "GetByName" and prints the public IPv4 address. **Anti-patterns:** * Not handling errors returned by the SDK calls. Always check for errors and handle them appropriately. * Using hardcoded server type or image names. Use the SDK's methods to retrieve available server types and images dynamically. ## 3. Configuration Management ### Standards * **Do This:** Use configuration management tools like Ansible, Chef, or Puppet to automate server configuration and application deployment. * **Don't Do This:** Manually configure servers. This is time-consuming, error-prone, and difficult to maintain. ### Why Configuration Management Matters * **Consistency:** Ensures that all servers are configured in the same way. * **Automation:** Automates the process of configuring and deploying applications. * **Idempotency:** Ensures that configuration changes are only applied if necessary. * **Version Control:** Configuration code can be stored in version control, allowing for tracking changes and easy rollbacks. ### Ansible Example (for configuring a web server) """yaml --- - hosts: webservers become: true # Run tasks with elevated privileges tasks: - name: Install Nginx apt: name: nginx state: present notify: - Restart Nginx - name: Copy Nginx configuration file template: src: nginx.conf.j2 #Jinja2 template dest: /etc/nginx/nginx.conf notify: - Restart Nginx - name: Enable Nginx site file: src: /etc/nginx/sites-available/default dest: /etc/nginx/sites-enabled/default state: link notify: - Restart Nginx handlers: - name: Restart Nginx systemd: name: nginx state: restarted """ "nginx.conf.j2" (example): """nginx user www-data; worker_processes auto; pid /run/nginx.pid; include /etc/nginx/modules-enabled/*.conf; events { worker_connections 768; } http { sendfile on; tcp_nopush on; tcp_nodelay on; keepalive_timeout 65; types_hash_max_size 2048; include /etc/nginx/mime.types; default_type application/octet-stream; ssl_protocols TLSv1.2 TLSv1.3; # Enforce TLS 1.2 or higher ssl_prefer_server_ciphers on; access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; gzip on; gzip_disable "msie6"; include /etc/nginx/conf.d/*.conf; server { listen 80; listen [::]:80; server_name {{ ansible_hostname }}; # Use Ansible fact root /var/www/html; index index.html index.htm index.nginx-debian.html; location / { try_files $uri $uri/ =404; } } } """ **Explanation:** The Ansible playbook installs Nginx, copies the Nginx configuration file to the target servers using Jinja2 templating, and enables the default site. The handler restarts Nginx whenever the configuration changes. The "server_name" is dynamically set based on the host's hostname retrieved by ansible. The nginx configuration also enforces TLS 1.2 or higher for enhanced security. **Anti-patterns:** * Not using templates for configuration files. Templates allow for dynamic configuration based on server-specific variables. * Not using handlers to restart services after configuration changes. This can lead to services running with outdated configurations. * Lack of security considerations. Storing secrets in plaintext configuration files. ## 4. Monitoring and Logging ### Standards * **Do This:** Implement comprehensive monitoring and logging for your Hetzner Cloud resources using tools like Prometheus, Grafana, Loki, and the ELK stack (Elasticsearch, Logstash, Kibana). Consider using Hetzner's Robot monitoring service for basic server health. * **Don't Do This:** Rely solely on manual checks or the Hetzner Cloud Console for monitoring. This provides limited visibility and doesn't scale. ### Why Monitoring and Logging Matter * **Early Detection of Issues:** Monitoring allows you to detect issues before they impact users. * **Troubleshooting:** Logging provides valuable information for diagnosing and resolving problems. * **Performance Optimization:** Monitoring and logging can help you identify performance bottlenecks and optimize your applications and infrastructure. * **Security Auditing:** Logs can be used to track security events and identify potential threats. ### Prometheus and Grafana Example 1. **Install Prometheus and Grafana:** Deploy Prometheus and Grafana to a Hetzner Cloud server or use a managed Prometheus/Grafana service. 2. **Install Node Exporter:** On each server you want to monitor, install the Node Exporter, which collects system metrics. """bash wget https://github.com/prometheus/node_exporter/releases/download/v1.7.0/node_exporter-1.7.0.linux-amd64.tar.gz tar xvfz node_exporter-1.7.0.linux-amd64.tar.gz sudo mv node_exporter-1.7.0.linux-amd64/node_exporter /usr/local/bin/ sudo useradd -rs /bin/false node_exporter sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter """ Create a systemd service file: "/etc/systemd/system/node_exporter.service" """ [Unit] Description=Node Exporter Wants=network-online.target After=network-online.target [Service] User=node_exporter Group=node_exporter Type=simple ExecStart=/usr/local/bin/node_exporter [Install] WantedBy=multi-user.target """ Enable and start the service: """bash sudo systemctl enable node_exporter sudo systemctl start node_exporter """ 3. **Configure Prometheus:** Configure Prometheus to scrape metrics from the Node Exporter on each server. Add the following to your "prometheus.yml" file: """yaml scrape_configs: - job_name: 'node_exporter' static_configs: - targets: ['<server_ip>:9100'] #Replace with server IP labels: environment: production #Optional label """ 4. **Import Grafana Dashboard:** Import a pre-built Grafana dashboard for Node Exporter (e.g., Node Exporter Full) to visualize the metrics. ### Loki Example Use Promtail to ship logs to Loki. 1. Install Promtail """bash wget https://github.com/grafana/loki/releases/download/v2.10.0/promtail-linux-amd64.zip unzip promtail-linux-amd64.zip chmod +x promtail-linux-amd64 mv promtail-linux-amd64 /usr/local/bin/promtail """ 2. Create "/etc/promtail.yaml" and point promtail to the loki instance """yaml server: http_listen_port: 9080 grpc_listen_port: 0 positions: filename: /tmp/positions.db clients: - url: http://<loki_ip>:3100/loki/api/v1/push scrape_configs: - job_name: system static_configs: - targets: - localhost labels: job: syslog __path__: /var/log/syslog """ 3. Create a systemd service file: "/etc/systemd/system/promtail.service" """ [Unit] Description=Promtail After=network.target [Service] User=root ExecStart=/usr/local/bin/promtail -config.file=/etc/promtail.yaml [Install] WantedBy=multi-user.target """ 4. Enable and Start the Promtail Service """bash sudo systemctl enable promtail sudo systemctl start promtail """ **Explanation:** This example configures Prometheus to collect system metrics from Node Exporter on each server and visualizes them in Grafana. It also sets up Promtail to send logs to Loki. Grafana can then query logs from Loki. **Anti-patterns:** * Not setting up alerting. Configure alerts in Prometheus or Grafana to notify you when critical metrics exceed thresholds. * Collecting too much data. Identify the key metrics and logs that are essential for monitoring and troubleshooting. ## 5. Security Best Practices ### Standards * **Do This:** Implement strong security practices for all Hetzner Cloud resources. This includes using strong passwords, enabling firewalls, regularly updating software, and implementing intrusion detection and prevention systems. * **Don't Do This:** Use default passwords, disable firewalls, or neglect security updates. ### Hardening Servers * **Firewall:** Configure firewalls (e.g., "ufw" or "iptables") to only allow necessary traffic. Only open ports that are required for your applications. * **SSH:** Disable password-based SSH authentication and use SSH keys instead. Configure SSH to listen on a non-standard port and disable root login. Update SSH regularly. * **Regular Updates:** Keep your operating system and software packages up to date with the latest security patches. * **Intrusion Detection/Prevention:** Consider using intrusion detection and prevention systems (IDS/IPS) to monitor your servers for malicious activity. Fail2ban is a simple option for preventing brute-force attacks. * **Secrets Management:** Employ secrets management solutions like HashiCorp Vault to securely store and manage sensitive information, such as API keys and passwords. Never hardcode secrets in your code or configuration files. ### Example Firewall Configuration (using "ufw") """bash # Enable UFW sudo ufw enable # Allow SSH (ensure your SSH port is open before enabling UFW) sudo ufw allow 22/tcp #Replace with your SSH port # Allow HTTP (port 80) sudo ufw allow 80/tcp # Allow HTTPS (port 443) sudo ufw allow 443/tcp # Deny all other incoming traffic sudo ufw default deny incoming sudo ufw default allow outgoing #Show firewall status sudo ufw status verbose """ **Explanation:** The example configures "ufw" to allow SSH, HTTP, and HTTPS traffic while denying all other incoming traffic. Remember to replace "22" with the actual SSH port if you have changed it from the default. **Anti-patterns:** * Using weak passwords or default credentials. * Disabling firewalls or not properly configuring them. * Not keeping software up to date with the latest security patches. * Storing secrets in plaintext. ## 6. Continuous Integration and Continuous Delivery (CI/CD) ### Standards * **Do This:** Implement CI/CD pipelines to automate the build, test, and deployment of your applications. Consider using tools like Jenkins, GitLab CI, GitHub Actions, or CircleCI. * **Don't Do This:** Manually build, test, and deploy your applications. This is time-consuming, error-prone, and difficult to scale. ### Why CI/CD Matters * **Automation:** Automates the build, test, and deployment process. * **Faster Releases:** Enables faster and more frequent releases. * **Improved Quality:** Automated testing helps to ensure that code changes are of high quality. * **Reduced Risk:** Automated deployments reduce the risk of errors. ### GitLab CI Example """yaml stages: - build - test - deploy build: stage: build image: docker:latest services: - docker:dind before_script: - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY script: - docker build -t "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA" . - docker push "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA" tags: - docker test: stage: test image: python:3.9 before_script: - pip install -r requirements.txt script: - pytest tags: - docker deploy: stage: deploy image: docker:latest services: - docker:dind before_script: - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY - apk add -U openssh-client - mkdir -p ~/.ssh - echo "$SSH_PRIVATE_KEY" | tr -d '\r' > ~/.ssh/id_rsa - chmod 600 ~/.ssh/id_rsa - ssh-keyscan <server_ip> >> ~/.ssh/known_hosts - ssh -o StrictHostKeyChecking=no <user>@<server_ip> "docker stop <container_name> || true && docker rm <container_name> || true" #Replace placeholders script: - ssh -o StrictHostKeyChecking=no <user>@<server_ip> "docker pull $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA && docker run -d -p 80:80 --name <container_name> $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA" #Replace placeholders tags: - docker only: - main """ **Explanation:** This GitLab CI configuration defines three stages: build, test, and deploy. The build stage builds a Docker image and pushes it to the GitLab registry. The test stage runs unit tests. The deploy stage deploys the Docker image to a Hetzner Cloud server via SSH. The "SSH_PRIVATE_KEY" variable should be stored as a secret in GitLab CI. The "only: - main" directive ensures that deployments only happen from the "main" branch. Remember to replace the placeholders with your actual values. **Anti-patterns:** * Not using CI/CD pipelines. * Not automating testing. * Storing secrets in the CI/CD configuration file. ## 7. Hetzner Specific Optimizations ### Standards * **Do This:** Utilize Hetzner-specific features and optimizations to improve performance and reduce costs. * **Don't Do This:** Ignore Hetzner-specific features and optimizations. ### Examples * **Placement Groups:** Use placement groups to colocate servers that need low latency communication. This can improve the performance of distributed applications. * **Private Networks:** Use private networks to isolate your resources and reduce network costs within the Hetzner Cloud environment. Internal traffic within the same private network does not impact external bandwidth usage. * **Load Balancers:** Utilize Hetzner Cloud Load Balancers for distributing traffic across multiple servers. Load balancers can improve availability and scalability of your applications. * **Storage Boxes:** Use Hetzner Storage Boxes for cost-effective object storage ### Placement Group Example (Terraform) """terraform resource "hcloud_placement_group" "pg-example" { name = "my_placement_group" type = "spread" # Or "strict" } resource "hcloud_server" "server1" { name = "web-server-03" server_type = "cx11" image = "ubuntu-22.04" location = "fsn1" placement_group_id = hcloud_placement_group.pg-example.id } resource "hcloud_server" "server2" { name = "web-server-04" server_type = "cx11" image = "ubuntu-22.04" location = "fsn1" placement_group_id = hcloud_placement_group.pg-example.id } """ **Explanation:** The code creates a placement group and then creates two servers and assigns them to that placement group, ensuring they are placed on different physical hosts (spread type). The "strict" placement group type may return an error during server creation if the placement constraint cannot be satisfied. **Anti-patterns:** * Not using placement groups when low latency communication is required. * Not using private networks to isolate resources. * Not leveraging the load balancer to improve application availability. ## 8. Contributing Back to the Community ### Standards: * **Do This**: Actively participate in the Hetzner community by sharing knowledge, contributing to open-source projects, and providing feedback to Hetzner. * **Don't Do This**: Be a passive consumer of the Hetzner ecosystem. ### Ways to Contribute * **Open Source**: Contribute to existing open-source projects related to Hetzner or create your own. Share Terraform modules, Ansible roles, SDK extensions, or other tools and libraries that you develop. * **Documentation**: Improve the official Hetzner documentation by submitting pull requests or bug reports. * **Community Forums**: Answer questions on the Hetzner community forums and help other users. * **Blog Posts and Tutorials:** Write blog posts and tutorials sharing your experiences and best practices for using Hetzner. * **Feedback**: Provide feedback to Hetzner on their products and services. Submit feature requests and bug reports. By adhering to these standards, development teams can ensure they are building reliable, scalable, and secure applications and infrastructure using Hetzner Cloud. This document serves as a living guide and should be updated as new features and best practices emerge within the Hetzner ecosystem.