# Performance Optimization Standards for Nim
This document outlines coding standards and best practices specifically designed for performance optimization in Nim. Adhering to these standards will result in faster, more responsive, and resource-efficient applications. These standards apply to the latest version of Nim and emphasize modern approaches.
## 1. Algorithmic Efficiency and Data Structures
### 1.1. Algorithm Selection
* **Do This:** Choose algorithms with optimal time complexity for the task at hand. Understand the trade-offs between different algorithms (e.g., sorting algorithms, search algorithms).
* **Don't Do This:** Blindly use the first algorithm that comes to mind. Ignore the impact of algorithm choice on performance, especially with large datasets.
**Why:** Choosing the right algorithm can dramatically impact performance. For example, using a bubble sort (O(n^2)) instead of a merge sort (O(n log n)) on a large dataset will lead to significantly slower execution.
**Example:**
"""nim
import algorithm
# Good: Using timSort (hybrid merge sort/insertion sort) in the standard library
let data = [3, 1, 4, 1, 5, 9, 2, 6]
var sortedData = data
sortedData.sort() # Uses timSort
# Bad: Implementing a naive (and slow) bubble sort
proc bubbleSort(data: var seq[int]) =
let n = data.len
for i in 0.. data[j+1]:
swap data[j], data[j+1]
var sortedDataBubble = data
bubbleSort(sortedDataBubble) # Extremely slow for large datasets
"""
### 1.2. Data Structure Optimization
* **Do This:** Select the most appropriate data structure for the task. Consider factors like access patterns, insertion/deletion frequency, and memory usage. Leverage Nim's rich standard library of data structures.
* **Don't Do This:** Use a data structure that doesn't fit the problem's requirements, leading to unnecessary overhead and poor performance. Overuse generic "seq" for everything.
* **Why:** Using the wrong data structure can cause significant performance bottlenecks. For example, using a list for frequent random access is inefficient compared to using an array.
**Example:**
"""nim
import tables, sets
# Good: Using a Table for key-value lookups with potential for string keys
var nameToAge = initTable[string, int]()
nameToAge["Alice"] = 30
nameToAge["Bob"] = 25
# Good: Using a Set to efficiently check for membership
var seenNumbers = initSet[int]()
seenNumbers.incl(10)
if 10 in seenNumbers:
echo "10 has been seen"
# Bad: Using a seq to check if keys exist or perform fast lookups.
var tempSeq: seq[(string, int)]
# imagine filling tempSeq, but using "for (key, value) in tempSeq: if key == "Alice": ..." is extremely slow
"""
### 1.3 Memory Management
* **Do This:** Understand Nim's memory management options (GC, ARC, ORC) and choose the most suitable one for your application. Consider using "{.push gcsafe.}" and "{.pop.}" pragmas for performance-critical sections to prevent GC pauses. Prefer value types (copy semantics) where applicable to minimize heap allocations. Use object variants and enums.
* **Don't Do This:** Rely solely on the default GC without considering its impact on latency. Create excessive temporary objects, increasing GC pressure.
* **Why:** Nim's memory management system has a direct impact on performance, especially in long-running or real-time applications. Minimizing memory allocations and deallocations reduces GC overhead. The new ORC GC is designed for efficiency, especially with destructors and ownership.
* **ORC GC (Latest Nim):** The ORC (Ownership and Reference Counting) GC in newer Nim versions (>= 2.0) manages memory using a combination of ownership tracking and reference counting. It's designed to minimize GC pauses and improve deterministic memory management.
**Example:**
"""nim
# Good: Using ARC/ORC for deterministic memory management
# Nim compiler options: -d:arc or -d:orc
proc processData(data: seq[int]): seq[int] =
# Minimize allocations within the loop
var result: seq[int]
for item in data:
result.add(item * 2)
return result
# Good: using GC Safe sections
proc someCalculation(data: seq[int]) : seq[int] {.gcsafe.} =
# no new memory allocations here, everything is on the stack.
return data
# Bad: Excessive allocation in a tight loop (causes frequent GC pauses)
proc processDataBad(data: seq[int]): seq[int] =
var result: seq[int]
for item in data:
result.add(new int(item * 2)) # Creates a new heap-allocated int for each item.
"""
### 1.4 String Handling
* **Do This:** Use "static[string]" for string literals that are known at compile time. Use "cstring" for interacting with C libraries. Pre-allocate strings of known size with "newString(size)". Use the "strutils" module efficiently.
* **Don't Do This:** Perform unnecessary string copies in loops. Use string concatenation ("&") excessively within performance-critical loops, as each concatenation creates a new string. Underestimate the impact of string operations on performance.
* **Why:** Strings are immutable in Nim, so concatenating or modifying them creates new string objects, which can be expensive. CStrings bypass the Nim memory manager entirely.
**Example:**
"""nim
import strutils
# Good: Using StringBuilder for efficient string concatenation within a loop
proc buildString(count: int): string =
var sb = newStringOfCap(count * 2)
for i in 0..= 2.0)
"""nim
import profile
proc slowFunction() =
var x = 0
for i in 0..1000000:
x += i
proc main() =
startProfile()
slowFunction()
stopProfile()
saveProfile("profile.data") # then use "nim profiler profile.data"
main()
"""
### 5.2. Benchmarking
* **Do This:** Use benchmarking tools to measure the performance of your code changes. Use the "benchmarks" module in Nim to create reproducible benchmarks.
* **Don't Do This:** Rely on anecdotal evidence to assess performance improvements.
* **Why:** Benchmarking provides quantitative data to verify the effectiveness of your optimizations changes, and catches regressions.
**Example:**
"""nim
import benchmarks
benchmark "String concatenation":
var s = ""
for i in 0..<1000:
s &= "x"
benchmark "StringBuilder":
var sb = newStringOfCap(1000)
for i in 0..<1000:
sb.add("x")
discard sb.toString
runBenchmarks()
"""
## 6. System-Level Considerations
### 6.1. I/O Optimization
* **Do This:** Use buffered I/O to reduce the number of system calls. Minimize disk access by caching frequently used data in memory. Use asynchronous I/O where appropriate.
* **Don't Do This:** Perform unbuffered I/O for large data transfers. Make excessive disk accesses.
* **Why:** I/O operations are often a performance bottleneck, so optimizing them can significantly improve application performance.
**Example:**
"""nim
# Good: Using buffered I/O
import streams
proc readFileBuffered(filename: string): string =
let fileStream = newFileStream(filename, fmRead)
defer: fileStream.close()
# Wrap the file stream with a buffered stream
let bufferedStream = newBufferedStream(fileStream)
return bufferedStream.readAll() # Efficiently reads the entire file
# Bad: Reading a file character by character without buffering
proc readFileUnbuffered(filename: string): string =
let fileStream = newFileStream(filename, fmRead)
defer: fileStream.close()
var result = ""
while not fileStream.atEnd():
result.add(fileStream.readChar()) # Inefficient: makes a system call for each character
return result
"""
### 6.2. Memory Usage
* **Do This:** Monitor your application's memory usage to identify potential memory leaks or excessive memory consumption. Use tools like Valgrind (Linux) to detect memory errors. Consider using data compression techniques to reduce memory footprint.
* **Don't Do This:** Ignore memory usage patterns. Allocate large amounts of memory without reason.
* **Why:** High memory usage can lead to reduced performance, crashes, or even denial-of-service. Leaked memory might cause even bigger issues down the road.
## 7. Specific Nim Features and Libraries
### 7.1 Zero-Cost Iterators
* **Do This:** When appropriate, write custom iterators rather than generating sequences with "map" or similar. These iterators perform the computation during the loop, rather than computing the complete sequence in-memory first.
* **Don't Do This:** Always generate sequences, especially large ones, before looping, wasting memory and compute cycles.
**Example**
"""nim
# Good: using a custom iterator
iterator numbers(n: int): int =
for i in 0..
danielsogl
Created Mar 6, 2025
This guide explains how to effectively use .clinerules
with Cline, the AI-powered coding assistant.
The .clinerules
file is a powerful configuration file that helps Cline understand your project's requirements, coding standards, and constraints. When placed in your project's root directory, it automatically guides Cline's behavior and ensures consistency across your codebase.
Place the .clinerules
file in your project's root directory. Cline automatically detects and follows these rules for all files within the project.
# Project Overview project: name: 'Your Project Name' description: 'Brief project description' stack: - technology: 'Framework/Language' version: 'X.Y.Z' - technology: 'Database' version: 'X.Y.Z'
# Code Standards standards: style: - 'Use consistent indentation (2 spaces)' - 'Follow language-specific naming conventions' documentation: - 'Include JSDoc comments for all functions' - 'Maintain up-to-date README files' testing: - 'Write unit tests for all new features' - 'Maintain minimum 80% code coverage'
# Security Guidelines security: authentication: - 'Implement proper token validation' - 'Use environment variables for secrets' dataProtection: - 'Sanitize all user inputs' - 'Implement proper error handling'
Be Specific
Maintain Organization
Regular Updates
# Common Patterns Example patterns: components: - pattern: 'Use functional components by default' - pattern: 'Implement error boundaries for component trees' stateManagement: - pattern: 'Use React Query for server state' - pattern: 'Implement proper loading states'
Commit the Rules
.clinerules
in version controlTeam Collaboration
Rules Not Being Applied
Conflicting Rules
Performance Considerations
# Basic .clinerules Example project: name: 'Web Application' type: 'Next.js Frontend' standards: - 'Use TypeScript for all new code' - 'Follow React best practices' - 'Implement proper error handling' testing: unit: - 'Jest for unit tests' - 'React Testing Library for components' e2e: - 'Cypress for end-to-end testing' documentation: required: - 'README.md in each major directory' - 'JSDoc comments for public APIs' - 'Changelog updates for all changes'
# Advanced .clinerules Example project: name: 'Enterprise Application' compliance: - 'GDPR requirements' - 'WCAG 2.1 AA accessibility' architecture: patterns: - 'Clean Architecture principles' - 'Domain-Driven Design concepts' security: requirements: - 'OAuth 2.0 authentication' - 'Rate limiting on all APIs' - 'Input validation with Zod'
# API Integration Standards for Nim This document provides comprehensive coding standards for API integration in Nim. It covers best practices for connecting with backend services and external APIs, focusing on maintainability, performance, and security. This guide assumes familiarity with basic Nim syntax and concepts. All examples are tailored for the latest Nim features and ecosystem libraries. ## 1. Architectural Considerations ### 1.1. Choosing an Architecture **Standard:** Select an architecture that aligns with the application's scalability, maintainability, and security requirements. Microservices, layered architectures, and hexagonal architectures are common choices. **Why:** The architectural choice significantly impacts how API integrations are handled. A well-defined architecture promotes modularity and testability while isolating API calls and minimizing dependencies of the core business logic to external dependencies. **Do This:** * **Microservices (Small-scale Integration):** Employ microservices with well-defined API contracts. * **Layered Architecture:** Abstract API interaction into a separate "Infrastructure" layer. * **Hexagonal Architecture:** Utilize ports and adapters to abstract API integrations. **Don't Do This:** * **Tight Coupling:** Avoid directly embedding API calls within the business logic. Tightly coupled systems become difficult to maintain and test. * **Monolithic Spaghetti Code:** An uncontrolled, sprawling codebase with no separation of concerns. **Example (Layered Architecture):** """nim # infrastructure/api_client.nim import httpclient proc getUserData(userId: string): Future[string] {.async.} = let client = newHttpClient() let url = "https://api.example.com/users/" & userId let response = await client.get(url) if response.status == Http200: return response.body else: raise newException(IOError, "API request failed: " & $response.status) # business/user_service.nim import infrastructure/api_client proc processUserData(userId: string): Future[string] {.async.} = try: let userData = await getUserData(userId) # Business logic to process user data return "Processed data: " & userData except IOError as e: echo "Error fetching user data: ", e.msg return "" # main.nim import business/user_service proc main() {.async.} = let result = await processUserData("123") echo result waitFor main() """ ### 1.2. Contract-First Approach **Standard:** Define API contracts using OpenAPI (Swagger) or similar specifications before implementation. Use code generation tools based on the OpenAPI specification. **Why:** A contract-first approach improves interoperability and reduces integration issues. It also facilitates parallel development by enabling front-end and back-end teams to work independently based on the agreed-upon contract. **Do This:** * Use OpenAPI/Swagger for API contract definitions. * Utilize code generation tools to produce Nim client code from OpenAPI definitions. **Don't Do This:** * **Ad-hoc API Design:** Creating APIs without documented contracts is difficult to maintain and integrate. * **Implicit Contracts:** Relying on existing API behavior without formal documentation can cause unexpected issues during updates. **Example (Generating Nim Client from OpenAPI):** 1. **Define OpenAPI Specification ("openapi.yaml"):** """yaml openapi: 3.0.0 info: title: User API version: 1.0.0 paths: /users/{userId}: get: summary: Get user by ID parameters: - in: path name: userId schema: type: string required: true description: ID of the user to retrieve responses: '200': description: Successful operation content: application/json: schema: type: object properties: id: type: string name: type: string """ 2. **Use a Code Generation Tool (e.g., "openapi-generator" with a Nim template. Assuming a docker container setup):** """bash docker run --rm -v ${PWD}:/local openapitools/openapi-generator-cli generate \ -i /local/openapi.yaml \ -g nim \ -o /local/generated """ 3. **Generated Code (Example):** The "generated/" folder will contain Nim modules representing API clients, models, and utilities. The exact structure will depend on the template used, but expect files such as "api.nim", "models.nim", and "client.nim". You would then "import generated/api". (Note: actual rendered generated code omitted for conciseness but follows this general structure). ## 2. HTTP Client Configuration ### 2.1. Use "httpclient" Library **Standard:** Utilize the "httpclient" library for making HTTP requests. **Why:** "httpclient" is a well-maintained and widely used library with robust features, including connection pooling, timeout settings, and TLS support. **Do This:** """nim import httpclient let client = newHttpClient() # Perform requests using client.get, client.post, etc. """ **Don't Do This:** * **Rolling Your Own HTTP Client:** Implementing a custom HTTP client is unnecessary and error-prone. * **Outdated Libraries:** Avoid using deprecated or unmaintained HTTP client libraries. ### 2.2. Configure Timeouts **Standard:** Set appropriate connection and request timeouts to prevent indefinite waiting and resource exhaustion. **Why:** Timeouts ensure that API calls do not hang indefinitely, improving application responsiveness and preventing resource leaks. **Do This:** """nim import httpclient let client = newHttpClient(timeout = 30.seconds, connectTimeout = 10.seconds) try: let response = await client.get("https://api.example.com") echo response.body except HttpRequestError as e: echo "Request timed out: ", e.msg finally: client.close() """ **Don't Do This:** * **Infinite Timeouts:** Allowing API calls to wait indefinitely. * **Unrealistic Timeouts:** Setting timeouts too short, leading to premature request failures. ### 2.3. Handle Errors Gracefully **Standard:** Implement proper error handling for API calls, including connection errors, HTTP status codes, and data parsing errors. **Why:** Graceful error handling ensures that the application remains stable and provides informative feedback to the user. **Do This:** """nim import httpclient let client = newHttpClient() try: let response = await client.get("https://api.example.com/nonexistent") if response.status == Http404: echo "Resource not found" elif response.status >= Http500: echo "Server error" else: echo response.body except HttpRequestError as e: echo "Connection error: ", e.msg finally: client.close() """ **Don't Do This:** * **Ignoring Errors:** Catching exceptions but not handling them properly. * **Generic Error Messages:** Providing vague or unhelpful error messages to the user. ### 2.4. Connection Pooling **Standard:** Leverage connection pooling to reuse existing connections and reduce overhead. **Why:** Connection pooling improves performance by avoiding the overhead of establishing a new connection for each API call. "httpclient" handles this automatically. **Do This:** """nim import httpclient let client = newHttpClient() # Connection pooling enabled by default for i in 1..10: let response = await client.get("https://api.example.com") echo response.body client.close() # Important - close when done. """ **Don't Do This:** * **Creating a New Client for Each Request:** Inefficient use of resources. * **Failing to Close the Client:** Can keep connections open longer than needed. ### 2.5. Use of "Future[T]" **Standard:** For asynchronous operations, return a "Future[T]" to encapsulate the eventual result. This enables non-blocking operation with "async/await". **Why:** Asynchronous API integrations are crucial for maintaining application responsiveness. "Future[T]" allows the caller to continue execution while the network request is in progress. **Do This:** """nim import httpclient proc fetchData(url: string): Future[string] {.async.} = let client = newHttpClient() try: let response = await client.get(url) return response.body except HttpRequestError as e: raise newException(IOError, "API request failed: " & e.msg) finally: client.close() proc main() {.async.} = let dataFuture = fetchData("https://api.example.com") echo "Doing other work..." let data = await dataFuture # Blocks until data is available echo "Data received: ", data waitFor main() """ **Don't Do This:** * **Blocking Calls in the Main Thread:** Leads to unresponsive applications. * **Ignoring Error Handling in Async Procedures:** Can result in unhandled exceptions. ## 3. Data Serialization and Deserialization ### 3.1. Choose a Serialization Format **Standard:** Select a suitable serialization format, typically JSON or Protocol Buffers (Protobuf). Consider performance, readability, and ecosystem support. **Why:** Choosing the right format can significantly affect the performance of API integrations and the ease of data manipulation. JSON is widely used because of its human-readability and broad support, while Protobuf offers better performance and schema validation at the cost of being less readable. **Do This:** * **JSON:** Use the "json" module from Nim's standard library or alternatives like "jester" for web applications. * **Protocol Buffers:** Use the "protobuf" library for high-performance scenarios and schema validation when appropriate. **Don't Do This:** * **Manual String Parsing:** Error-prone and inefficient. * **Unnecessary Complexity:** Avoid using over-complicated serialization formats when simpler options suffice. ### 3.2. JSON Handling **Standard:** Use the "json" module for JSON serialization and deserialization. Define Nim types that correspond to the JSON structure. **Why:** Type safety is essential for ensuring data integrity and preventing runtime errors. **Do This:** """nim import json type User = object id: string name: string email: string proc fromJson(jsonNode: JsonNode): User = User( id: jsonNode["id"].getStr(), name: jsonNode["name"].getStr(), email: jsonNode["email"].getStr() ) proc toJson(user: User): JsonNode = %* { "id": user.id, "name": user.name, "email": user.email } let jsonString = """ { "id": "123", "name": "John Doe", "email": "john.doe@example.com" } """ let jsonNode = parseJson(jsonString) let user = fromJson(jsonNode) echo user.name let userJson = toJson(user) echo userJson.pretty """ **Don't Do This:** * **Directly Accessing JSON Node Values:** Reduces type safety and increases the risk of runtime errors. * **Ignoring Missing Fields:** Handle cases where JSON fields are absent or null. ### 3.3. Protocol Buffers Handling **Standard:** Define Protobuf schemas using ".proto" files and compile them to Nim code using the Protobuf compiler. **Why:** Protobuf offers efficient serialization and schema validation. It's especially beneficial when dealing with large data sets or performance-critical applications. **Do This:** 1. **Define Protobuf Schema ("user.proto"):** """protobuf syntax = "proto3"; message User { string id = 1; string name = 2; string email = 3; } """ 2. **Compile Proto File (Requires the protobuf compiler and the Nim protobuf library):** """bash protoc --nim_out=. user.proto """ 3. **Use Generated Code:** """nim import user let user = User(id: "123", name: "John Doe", email: "john.doe@example.com") let serializedData = user.serialize() let deserializedUser = User() deserializedUser.deserialize(serializedData) echo deserializedUser.name """ **Don't Do This:** * **Manually Implementing Serialization:** Error-prone and inefficient. * **Ignoring Schema Updates:** Keep Protobuf schemas consistent between client and server. ## 4. Authentication and Authorization ### 4.1. Use Secure Authentication Methods **Standard:** Implement secure authentication methods such as OAuth 2.0 or API keys. Protect credentials and tokens. **Why:** Authentication ensures only authorized users or applications can access APIs, preventing unauthorized access and potential data breaches. **Do This:** * **OAuth 2.0:** Use OAuth 2.0 for delegated authorization. Use a library like "nim-oauth2". * **API Keys:** Generate unique API keys for each client and store them securely. **Don't Do This:** * **Basic Authentication over HTTP:** Transmits credentials in plain text. * **Hardcoding Credentials:** Stores credentials directly in the source code. ### 4.2. API Key Authentication **Standard:** Store API keys securely (e.g., environment variables or configuration files). Use HTTPS for transmitting API keys. **Do This:** """nim import httpclient, os proc callApiWithApiKey(url: string): Future[string] {.async.} = let apiKey = getEnv("API_KEY") # Retrieve from environment variable if apiKey.len == 0: raise newException(IOError, "API_KEY environment variable not set") let client = newHttpClient() defer: client.close() let headers = {"X-API-Key": apiKey} let response = await client.get(url, headers = headers) if response.status == Http200: return response.body else: raise newException(IOError, "API request failed: " & $response.status) proc main() {.async.} = try: let result = await callApiWithApiKey("https://api.example.com/data") echo result except IOError as e: echo "API Error:", e.msg waitFor main() """ **Don't Do This:** * **Storing API Keys in the Codebase:** Exposes API keys if the code is compromised. * **Transmitting API Keys over HTTP:** Allows eavesdropping and interception. ### 4.3. OAuth 2.0 Authentication **Standard:** Implement the OAuth 2.0 flow to obtain access tokens from the authorization server; Refresh the tokens as needed; Store tokens securely. Use a library like "nim-oauth2" to handle the complexities of the OAuth flow. **Why:** OAuth 2.0 enables user-delegated authorization, allowing applications to access resources on behalf of the users without storing their credentials. **Important:** The "nim-oauth2" library has not been updated in a while. Evaluate it thoroughly before using, considering potential security vulnerabilities or compatibility problems with modern OAuth 2.0 implementations. Pay close attention to the specific grant type implemented in "nim-oauth2", as these will vary among OAuth providers. You might need to adapt it or build on top of it. **Caveat:** Due to the potential complexity and lack of a fully mature and actively maintained OAuth2 library for Nim, it is essential to carefully evaluate existing solutions and possibly implement custom elements to align with current OAuth2 standards and specific provider requirements. **Do This:** """nim # Example skeleton - replace with real implementation using your OAuth2 provider. # This is just a conceptual example. import httpclient, os #NOTE: A real oauth2 library will provide secure and robust token management type OAuthToken = object accessToken: string refreshToken: string expiry: int # Unix timestamp var oauthToken: OAuthToken # Global or context specific. proc obtainAccessToken(): Future[OAuthToken] {.async.} = # Implement OAuth 2.0 flow (e.g., Authorization Code Grant) # Requires interaction with authorization server's endpoints. # Retrieve client ID, client secret, authorization URL, token URL # **IMPORTANT:** Handle code exchange, token storage and refresh tokens SECURELY ## This is a simplified example skeleton. let client = newHttpClient() defer: client.close() # let tokenEndpoint = ... # let response = await client.post(tokenEndpoint, ...) # Parse the response and return the token. ## THIS IS JUST A PLACEHOLDER. IMPLEMENT CORRECTLY! result = OAuthToken(accessToken: "dummy", refreshToken: "dummy", expiry: 0) proc callApiWithOAuth(url: string): Future[string] {.async.} = if oauthToken.accessToken.len == 0 or oauthToken.expiry < epochTime(): oauthToken = await obtainAccessToken() #Blocking call if token is expired. let client = newHttpClient() defer: client.close() let headers = {"Authorization": "Bearer " & oauthToken.accessToken} try: let response = await client.get(url, headers = headers) if response.status == Http200: return response.body elif response.status == Http401: # Token expired oauthToken = await obtainAccessToken() # Retry the request with the refreshed token (omitted for brevity). raise newException(IOError, "Token expired and was unable to refresh.") else: raise newException(IOError, "OAuth API request failed: " & $response.status) except HttpRequestError as e: raise newException(IOError, "HttpRequestError: " & e.msg) proc main() {.async.} = try: let result = await callApiWithOAuth("https://api.example.com/securedata") echo result except IOError as e: echo "OAuth API Error:", e.msg waitFor main() """ **Don't Do This:** * **Storing Tokens Insecurely:** Saving refresh tokens in plain text. * **Using Implicit Grant Flow:** The implicit grant is prone to security risks and generally deprecated. * **Not refreshing tokens:** Tokens expire and must be routinely refreshed. ## 5. Logging and Monitoring ### 5.1. Implement Logging **Standard:** Log API calls, including request and response data, errors, and performance metrics. **Why:** Logging provides valuable insights into API usage, helps identify performance bottlenecks, and facilitates debugging. **Do This:** """nim import logging, httpclient logging.basicConfig(level = DEBUG) # Set appropriate logging level. proc callApi(url: string): Future[string] {.async.} = let client = newHttpClient() defer: client.close() logging.debug("Making request to: " & url) let startTime = epochTime() try: let response = await client.get(url) if response.status == Http200: let endTime = epochTime() logging.info("Request to " & url & " completed in " & $(endTime - startTime) & " seconds.") logging.debug("Response body: " & response.body) return response.body else: logging.error("API request failed: " & $response.status) raise newException(IOError, "API request failed: " & $response.status) except HttpRequestError as e: logging.error("Connection error: " & e.msg) raise except Exception as e: logging.error("Unhandled error:" & e.msg) raise proc main() {.async.} = try: let result = await callApi("https://api.example.com/data") echo result except IOError as e: echo "API Error:", e.msg waitFor main() """ **Don't Do This:** * **Logging Sensitive Data:** Avoid logging confidential information (e.g., passwords, API keys). * **Ignoring Logging:** Makes it difficult to troubleshoot issues. ### 5.2. Implement Monitoring **Standard:** Monitor API performance, error rates, and resource usage. **Why:** Monitoring provides real-time visibility into the health of APIs and helps detect potential issues before they impact users. **Do This:** * Use metrics libraries to track request latency, error rates, and resource consumption. * Integrate with monitoring tools (e.g. Prometheus, Grafana) for visualization and alerting. **NOTE:** There are not commonly used metrics libraries mature options in the Nim ecosystem. Consider using external daemons to monitor performance or contributing to an existing library. ### 5.3 Rate limiting **Standard**: Enforce rate limits to protect APIs from abuse and prevent resource exhaustion. Implement client-side and server-side rate limiting mechanisms. **Why**: Rate limiting protects infrastructure by preventing excessive requests from a single user and potentially preventing denial-of-service attacks. **Do This**: """nim import httpclient, locks, times # Basic in-memory rate limiter (for demonstration purposes only, consider Redis or a similar solution for production) type RateLimiter = ref object maxRequests: int window: Duration requestCounts: Table[string, int] # Keyed by client identifier. For demonstrative simplicity, we use a simple "string". lastReset: Time lock: Lock proc newRateLimiter(maxRequests: int, window: Duration): RateLimiter = RateLimiter(maxRequests: maxRequests, window: window, requestCounts: initTable[string, int](), lastReset: now(), lock: newLock()) proc isAllowed(limiter: RateLimiter, clientIdentifier: string): bool = acquire(limiter.lock) defer: release(limiter.lock) let currentTime = now() if currentTime - limiter.lastReset >= limiter.window: limiter.requestCounts.clear() #limiter.lastReset = currentTime #Keep the "lastReset" to the first request of the new window, so the time is accurately tracked. limiter.lastReset = currentTime - (currentTime mod limiter.window).Duration #If the duration wasn't aligned, reset the duration. #echo "window expired" let requestCount = limiter.requestCounts.getOrDefault(clientIdentifier, 0) if requestCount < limiter.maxRequests: limiter.requestCounts[clientIdentifier] = requestCount + 1 return true else: return false proc callLimitedApi(url: string, clientIdentifier: string, limiter: RateLimiter): Future[string] {.async.} = if not limiter.isAllowed(clientIdentifier): raise newException(IOError, "Rate limit exceeded for client: " & clientIdentifier) let client = newHttpClient() defer: client.close() try: # Simulate API call let response = await client.get(url) if response.status == Http200: return response.body else: raise newException(IOError, "API request failed: " & $response.status) except HttpRequestError as e: raise newException(IOError, "API request failed: " & e.msg) proc main() {.async.} = let limiter = newRateLimiter(maxRequests = 5, window = 10.seconds) let clientIdentifier = "testClient" for i in 0..<10: try: let result = await callLimitedApi("https://api.example.com/data", clientIdentifier, limiter) echo "Request ", i, ": ", result await sleepAsync(100.milliseconds) # Simulate client behaviour. except IOError as e: echo "Request ", i, ": Error: ", e.msg waitFor main() """ **Don't Do This:** * **Ignoring Rate Limiting:** Leaves APIs vulnerable to abuse. * **Implementing Insecure Rate Limiting:** Rate limiting based on IP address, which can be easily circumvented. ## 6. Testing ### 6.1. Write Unit Tests **Standard:** Write unit tests for API integration logic, mocking external dependencies. **Why:** Unit tests isolate and verify the behavior of individual components, ensuring that the API integration logic works as expected. **Do This:** * Use a mocking library (e.g. "unittest2") to simulate API responses. * Test error handling, data parsing, and authentication logic. ### 6.2. Write Integration Tests **Standard:** Write integration tests to verify the end to end interactions between the application and external APIs. **Why:** Integration tests ensure that different components work together correctly. **Do This:** * Use real API endpoints or test environments. * Verify data consistency and error handling. ## 7. Security Considerations ### 7.1. Input Validation **Standard:** Sanitize and validate all input data received from external APIs. **Why:** Input validation prevents injection attacks (e.g. SQL injection, XSS) and ensures data integrity. **Do This:** * Use strong typing and schema validation * Escape or encode user-provided data before using in queries or responses. ### 7.2. Output Encoding **Standard:** Encode output data properly to prevent cross-site scripting (XSS) vulnerabilities. **Why:** Output encoding ensures that untrusted data cannot execute malicious code in the user's browser. ## Conclusion Following these API integration standards will lead to more maintainable, performant, and secure Nim applications. Remember that this document should be a living document and updated as Nim evolves. By paying attention to architectural considerations, error handling, security, and testing, you can build robust integrations.
# Core Architecture Standards for Nim This document outlines the core architecture standards for Nim projects. These guidelines promote maintainability, scalability, performance, and security. They're designed to be used by both human developers and AI coding assistants to ensure consistency and quality across the codebase. ## 1. Architectural Patterns ### 1.1. Layered Architecture **Do This:** * Organize your application into distinct layers, typically: * Presentation Layer (UI, CLI) * Application Layer (Business Logic, Use Cases) * Domain Layer (Entities, Value Objects) * Infrastructure Layer (Database, External Services, I/O) **Why:** Layered architecture decouples concerns, making the application easier to understand, test, and modify. Changes in one layer have minimal impact on others. **Example:** """nim # domain/user.nim type UserID = distinct int User = object id: UserID username: string email: string # application/user_service.nim import domain/user proc createUser(username, email: string): Result[User, string] = ## Creates a new user. if username.len < 3: return err("Username must be at least 3 characters long.") # ... other validation logic ... let newUser = User(id: UserID(hash(username)), username: username, email: email) # ... persist user (infrastructure layer) ... return ok(newUser) # presentation/cli.nim import application/user_service import std/terminal proc run(): void = echo "Enter username:" let username = readLine(stdin) echo "Enter email:" let email = readLine(stdin) case createUser(username, email) of ok(user): echo "User created with ID: ", user.id of err(errorMessage): echo "Error creating user: ", errorMessage run() """ **Don't Do This:** * Avoid tight coupling between layers. Don't let the presentation layer directly access the database. * Don't create God objects that handle multiple responsibilities across layers. ### 1.2. Hexagonal Architecture (Ports and Adapters) **Do This:** * Define clear boundaries around your core domain logic. * Interact with the outside world (databases, external services) through interfaces (ports). * Implement adapters to translate between the interface and the specific technology. **Why:** Hexagonal architecture makes the core domain independent of infrastructure details. Enables easier testing and swapping of external dependencies. **Example:** """nim # core/user_repository.nim (Port) import domain/user type UserRepository = object # Abstract interface proc findUser(repo: UserRepository, id: UserID): Result[User, string] {.raises: [Defect].} proc saveUser(repo: UserRepository, user: User): Result[void, string] {.raises: [Defect].} # infrastructure/postgres_user_repository.nim (Adapter) import core/user_repository import domain/user import std/db_postgres type PostgresUserRepository = object of UserRepository connection: PgConnection proc findUser(repo: PostgresUserRepository, id: UserID): Result[User, string] = # Implementation using Postgres try: let query = "SELECT id, username, email FROM users WHERE id = $1" let result = repo.connection.exec(query, $id) if result.ntuples == 0: return err("User not found.") return ok(User(id: UserID(result.getStr(0, "id").parseInt), username: result.getStr(0, "username"), email: result.getStr(0, "email"))) except PgError as e: return err(e.msg) proc saveUser(repo: PostgresUserRepository, user: User): Result[void, string] = # Implementation using Postgres try: let query = "INSERT INTO users (id, username, email) VALUES ($1, $2, $3)" repo.connection.exec(query, $user.id, user.username, user.email) return ok() except PgError as e: return err(e.msg) # application/user_service.nim import core/user_repository import domain/user proc createUser(repo: UserRepository, username, email: string): Result[User, string] = ## Creates a new user using provided repository let newUser = User(id: UserID(hash(username)), username: username, email: email) let saveResult = repo.saveUser(newUser) if saveResult.isErr(): return saveResult.mapErr(proc(x: string): string = "Failed to save user: " & x) return ok(newUser) """ **Don't Do This:** * Let infrastructure details leak into the core domain. * Create direct dependencies between domain logic and specific database implementations. ### 1.3 Microservices **Do This:** * Decompose large applications into smaller, independent services. Each service should have a specific responsibility. * Communicate between services using well-defined APIs (e.g., REST, gRPC, message queues). * Design each service to be independently deployable and scalable. * Embrace eventual consistency in distributed systems. **Why:** Microservices allow teams to work independently, scale specific parts of the application, and improve fault isolation. **Example:** * A "UserService" for managing user accounts. * A "ProductService" for managing product catalogs. * An "OrderService" for handling order placement and fulfillment. These services would communicate via REST APIs or a message queue like RabbitMQ. A sample UserService endpoint might be: """nim # UserService (simplified) import std/httpclient import std/json proc createUser(username, email: string): JsonNode = # ... create user logic ... result = %* {"id": userId, "username": username, "email": email} # OrderService (simplified) import std/httpclient import std/json proc placeOrder(userId, productId: int): JsonNode = let client = newHttpClient() let userResult = client.get("http://userservice/users/" & $userId) # Assuming a user service if userResult.status != Http200: raise newException(ValueError, "Could not retrieve user information") # ... continue order placing based on the retrieved user data result = %* {"orderId": someOrderId, "userId": userId, "productId": productId} """ **Don't Do This:** * Create tightly coupled services that depend on each other's internal implementation details. * Build a distributed monolith, where services are technically separate but still require coordinated deployments. ## 2. Project Structure and Organization ### 2.1. Package Management with Nimble **Do This:** * Use "nimble" to manage dependencies. * Create a "*.nimble" file for your project. * Specify dependencies clearly with version constraints. * Use "src/", "tests/", and "docs/" directories following the standard convention. **Why:** "nimble" ensures reproducible builds and simplifies dependency management. Consitent directory structures improve readability. **Example:** """nim # myproject.nimble version = "0.1.0" author = "Your Name" description = "A brief description of your project." license = "MIT" requires "nim >= 1.6.0" requires "chronos >= 1.4.0" srcDir = "src" binDir = "bin" testDir = "tests" # dependencies requires "https://github.com/username/other_library.git" """ **Don't Do This:** * Manually download and manage dependencies. * Check in dependencies directly into your repository. ### 2.2. Module Structure **Do This:** * Organize code into logical modules. * Use meaningful module names. * Limit the size of each module to a manageable scope (e.g., a single class or a set of related functions). * Use "import" statements to declare dependencies between modules. * Favor explicit exports to control the module's public API. **Why:** Modules promote code reuse, reduce complexity, and improve maintainability. **Example:** """nim # src/mymodule.nim type MyType* = object # Asterisk makes the type public field1*: int # Also public field2: string # Private proc myFunction*(x: int): int {.exportc.} = # Public & exported for C interop ## A public function. return x * 2 proc privateHelper(y: int): int = ## A private helper function. return y + 1 """ **Don't Do This:** * Create large, monolithic modules that contain unrelated code. * Expose internal implementation details through the public API. ### 2.3. Error Handling Strategy **Do This:** * Use "Result[T, E]" for functions that can fail. "Result" type requires explicit error handling in the caller. * Use exceptions sparingly, primarily for truly exceptional situations (e.g., out-of-memory). Use only for situations where the program *cannot* reasonably recover. * Consider discriminated unions (enums with associated data) for representing different error states. * Log errors with sufficient context for debugging. **Why:** Clear error handling improves the reliability of the application. "Result" types avoid silent failures. **Example:** """nim import std/net proc connectToHost(host: string, port: Port): Result[Socket, string] = try: let socket = newSocket(AF_INET, SOCK_STREAM, IPPROTO_TCP) socket.connect(host, port) return ok(socket) except OSError as e: return err(e.msg) let connectionResult = connectToHost("example.com", Port(80)) case connectionResult of ok(socket): echo "Connected successfully!" socket.close() of err(errorMessage): echo "Connection failed: ", errorMessage """ **Don't Do This:** * Ignore potential errors. * Rely solely on exceptions for all error handling (can make control flow difficult to follow). * Catch exceptions without logging or handling them appropriately. * Use exceptions for routine, expected failures. Handle expected failure states with "Result" or similar constructs. ## 3. Coding Conventions ### 3.1. Naming Conventions **Do This:** * Use camelCase for variables and parameters (e.g., "userName", "orderId"). * Use PascalCase for types and modules (e.g., "User", "OrderService"). * Use SCREAMING_SNAKE_CASE for constants (e.g., "MAX_RETRIES"). * Use descriptive and meaningful names. * Prefix boolean variables with "is", "has", or "can" (e.g., "isLoggedIn", "hasPermission"). **Why:** Consistent naming conventions improve code readability and help developers quickly understand the purpose of different entities. **Example:** """nim const MAX_CONNECTIONS = 10 type User = object userId: int userName: string emailAddress: string isLoggedIn: bool proc getUserById(userId: int): Result[User, string] = # Implementation here using camelCase for variables let userName = "exampleUser" let emailAddress = "user@example.com" return ok(User(userId: userId, userName: userName, emailAddress: emailAddress, isLoggedIn: true)) """ **Don't Do This:** * Use single-letter variable names (except for loop counters). * Use abbreviations that are unclear or ambiguous. * Violate the established naming conventions for Nim. ### 3.2. Immutability **Do This:** * Use "let" for variables that should not be reassigned. * Use immutable data structures when appropriate (e.g., "tuple", "string", "set", "frozenset"). * Avoid modifying data structures in place. Instead, create a copy with the desired modifications. **Why:** Immutability reduces the risk of unexpected side effects and makes code easier to reason about. It can also enable performance optimizations. **Example:** """nim type Point = tuple[x: int, y: int] # Immutable tuple let origin: Point = (x: 0, y: 0) proc movePoint(point: Point, dx, dy: int): Point = ## Returns a *new* Point, because Point is immutable. return (x: point.x + dx, y: point.y + dy) let newPoint = movePoint(origin, 10, 20) echo origin # (x: 0, y: 0) - unchanged echo newPoint # (x: 10, y: 20) """ **Don't Do This:** * Use "var" when "let" is sufficient. * Modify data structures in place when it's not necessary. ### 3.3. Code Formatting **Do This:** * Use 2-space indentation. * Keep lines short (ideally less than 120 characters). * Use blank lines to separate logical blocks of code. * Follow the official Nim style guide (see links at top). **Why:** Consistent code formatting improves readability and makes it easier to collaborate with other developers. **Example:** """nim proc calculateTotalPrice(items: seq[(string, float)], taxRate: float): float = ## Calculates the total price of a list of items with tax. var subtotal = 0.0 for item in items: subtotal += item[1] let taxAmount = subtotal * taxRate return subtotal + taxAmount """ **Don't Do This:** * Use inconsistent indentation. * Write excessively long lines of code. * Omit blank lines between logical blocks. ### 3.4 Concurrency and Parallelism **Do This:** * Use "channels" and "threads" from "std/channels" and "std/threadpool" for concurrent operations. * Use "async" and "await" for asychronous operation using the "asyncdispatch" library. * Minimize shared mutable state. If shared state is unavoidable, use locks or other synchronization primitives to protect it. * Use "spawn" to create lightweight threads ("agents"). * Consider using a task queue or pool for managing concurrent tasks. Why: Using concurrency and parallelism efficiently improves the performance and responsiveness of the application. Following proper guidelines prevents race conditions and deadlocks. Example: """nim # Example using async/await with asyncdispatch import asyncdispatch import std/net import std/strutils # for split async proc handleClient(socket: Socket): Future[void] = defer: socket.close() var buffer: array[4096, char] let bytesRead = await socket.recv(buffer.addr, buffer.len) if bytesRead > 0: let request = $buffer[0..<bytesRead] echo "Received: ", request let parts = request.split("\r\n") let response = "HTTP/1.1 200 OK\r\nContent-Length: 12\r\n\r\nHello World!" await socket.send(response.cstring, response.len) async proc startServer(port: Port): Future[void] = let server = newAsyncSocket() server.bindAddr(port) server.listen() echo "Server listening on port ", port while true: let client = await server.accept() discard handleClient(client) # Detach the handler; no need to await proc main(): void = asyncCheck run(startServer(Port(8080))) # Start the server and check errors main() """ **Don't Do This:** * Share mutable data between threads without proper synchronization. * Create too many threads, as this can lead to excessive context switching and reduced performance. * Block the main thread with long-running operations. * Ignore potential exceptions or errors in concurrent code. ## 4. Testing ### 4.1. Unit Testing **Do This:** * Write unit tests for all critical functions and modules. * Use a testing framework like "unittest2". * Follow the Arrange-Act-Assert pattern. * Strive for high test coverage. **Why:** Unit tests verify the correctness of individual components and prevent regressions. **Example:** """nim import unittest2 import src/mymodule suite "MyModule Tests": test "myFunction returns the correct result": check myFunction(5) == 10 test "Can create MyType": let myInstance = MyType(field1: 1, field2: "test") check myInstance.field1 == 1 check myInstance.field2 == "test" runAllSuites() """ **Don't Do This:** * Skip writing unit tests. * Write tests that are too complex or tightly coupled to implementation details. * Ignore failing tests. ### 4.2. Integration Testing **Do This:** * Write integration tests to verify the interaction between different components. * Test the application's integration with external services and databases. * Use mocks or stubs to isolate components during testing. **Why:** Integration tests ensure that different parts of the application work together correctly. ### 4.3. End-to-End Testing **Do This:** * Write end-to-end tests to verify the entire application workflow from the user's perspective. * Use a testing framework like "karax" or "selenium" to automate browser interactions. * Run end-to-end tests in a dedicated testing environment. **Why:** End-to-end tests ensure that the application meets the overall requirements and provides a satisfactory user experience. ## 5. Documentation ### 5.1. API Documentation **Do This:** * Use doc comments ("##") to document all public types, procedures, and variables. * Include a brief description of the purpose, parameters, and return values. * Use the "#[...]" pragma to add metadata about the code. * Generate API documentation using "nim doc". **Why:** API documentation makes it easier for other developers to use and understand the code. **Example:** """nim ## Represents a user account. type User* = object ## The user's unique identifier. id*: int ## The user's username. username*: string ## The user's email address. email*: string ## Retrieves a user by their ID. ## ## Parameters: ## userId: The ID of the user to retrieve. ## ## Returns: ## The user with the given ID, or "nil" if no such user exists. proc getUserById*(userId: int): User {.raises: [IOError].} = # Implementation here discard """ **Don't Do This:** * Omit documentation for public API elements. * Write vague or incomplete documentation. * Fail to keep documentation up-to-date as the code changes. ### 5.2. Code Comments **Do This:** * Use comments to explain complex or non-obvious code. * Focus on explaining the *why* rather than the *what*. * Keep comments concise and relevant. **Why:** Code comments make it easier for other developers to understand the code's logic and intent. **Don't Do This:** * Comment on every line of code. * Write comments that are redundant or obvious. * Let comments become stale or inaccurate. ### 5.3 Usage Examples **Do This:** * Include basic, runnable usage examples. * Focus on the 'happy path' to show how to use code. * Keep examples short. **Why:** Usage examples will allow quick start to developers, and demonstrate expected use of the code. **Example** """nim # Example usage: import mymodule let result = myFunction(5) echo "Result: ", result # Output: Result: 10 """ This detailed document provides comprehensive development standards for Nim, considering modern approaches and best practices. By adhering to these guidelines, your Nim development team will be well-equipped to build robust, maintainable, and scalable applications.
# Component Design Standards for Nim This document outlines the coding standards for component design in Nim. It aims to provide guidelines for creating reusable, maintainable, and efficient components. These standards leverage Nim's unique features and best practices to ensure high-quality code. The recommendations are based on the latest Nim version and current best practices. ## 1. General Principles ### 1.1. Abstraction * **Do This:** Design components with well-defined interfaces. Hide implementation details behind these interfaces. """nim # Good: Abstract interface type SoundPlayer = interface proc play(filename: string) {.abstract.} type MP3Player* = object of SoundPlayer # Implementation details hidden proc play(filename: string) {.override.} = echo "Playing MP3: " & filename """ * **Don't Do This:** Expose internal data structures or implementation details directly. This leads to tight coupling and makes refactoring difficult. """nim # Bad: Exposing internal data type Database = object connectionString: string # Exposed! """ * **Why:** Abstraction reduces dependencies, making components easier to test, maintain, and reuse. It allows you to change the internal workings of a component without affecting its clients. ### 1.2. Single Responsibility Principle (SRP) * **Do This:** Ensure each component has one specific responsibility. """nim # Good: Separate concerns type EmailSender = object # Handles email sending only proc sendEmail(to: string, subject: string, body: string) type UserValidator = object # Handles user validation only proc validateUser(user: User): bool """ * **Don't Do This:** Create components that handle multiple unrelated tasks. This makes the code harder to understand and modify. """nim # Bad: Multiple responsibilities type UserManager = object # Handles both user management AND email sending proc createUser(user: User) proc sendEmail(to: string, subject: string, body: string) """ * **Why:** SRP improves code clarity and maintainability. When a component only has one responsibility, it's easier to understand, test, and modify. Changes to one responsibility don't affect other parts of the system. ### 1.3. Loose Coupling * **Do This:** Minimize the dependencies between components. Use interfaces, events, and message passing to decouple components. """nim # Good: Loose coupling with interfaces type Logger = interface proc log(message: string) {.abstract.} type ConsoleLogger* = object of Logger proc log(message: string) {.override.} = echo message type FileLogger* = object of Logger filename: string proc log(message: string) {.override.} = writeFile(filename, message & "\n") proc doSomething(logger: Logger) = logger.log("Doing something...") """ * **Don't Do This:** Create tight dependencies between components. This makes it difficult to change or replace one component without affecting others. """nim # Bad: Tight coupling type ComponentA = object b: ComponentB # Directly depends on ComponentB proc doSomething(a: ComponentA) = a.b.someMethod() """ * **Why:** Loose coupling makes components more independent and reusable. Changes in one component are less likely to affect others. This improves the overall flexibility and maintainability of the system. ### 1.4. Cohesion * **Do This:** Ensure the elements within a component are related and work together toward a common purpose. """nim # Good: High cohesion type MathUtils = object # Contains methods related to mathematical operations proc add(a, b: int): int proc subtract(a, b: int): int """ * **Don't Do This:** Create components with unrelated elements or responsibilities. This indicates a lack of cohesion and makes the component harder to understand and maintain. """nim # Bad: Low cohesion type UtilityFunctions = object # Contains unrelated functions proc add(a, b: int): int proc formatDate(date: DateTime): string """ * **Why:** High cohesion makes components easier to understand and maintain. When the elements within a component are related, it's easier to reason about the component's behavior. ### 1.5. Dependency Injection * **Do This:** Inject dependencies into components rather than creating them internally. """nim # Good: Dependency Injection type UserService = object repository: UserRepository # Dependency injected proc registerUser(service: UserService, user: User) = service.repository.save(user) proc createUserService(repository: UserRepository): UserService = return UserService(repository: repository) """ * **Don't Do This:** Create dependencies directly within components. This makes testing and reuse more difficult, as it requires mocking global objects. """nim # Bad: Hardcoded dependency type UserService = object repository: UserRepository = UserRepository() # Hardcoded dependency proc registerUser(service: UserService, user: User) = service.repository.save(user) """ * **Why:** Dependency injection improves testability, maintainability, and reusability. It allows you to easily replace dependencies with mocks or different implementations. ## 2. Component Types ### 2.1. Pure Functions * **Do This:** Favor pure functions whenever possible. Pure functions have no side effects and always return the same output for the same input. """nim # Good: Pure function proc add(a, b: int): int = # No side effect, returns the same output for same input return a + b """ * **Don't Do This:** Use functions with side effects when a pure function would suffice. Side effects can make code harder to understand and test. """nim # Bad: Function with side effect var sum: int = 0 proc addToSum(a: int) = sum += a # Side effect: modifies global variable """ * **Why:** Pure functions are predictable, testable, and easy to reason about. They can be easily parallelized and optimized by the compiler and the runtime. ### 2.2. Objects and Classes * **Do This:** Use objects and classes to encapsulate data and behavior. """nim # Good: Object with encapsulated data and behavior type Person* = object name*: string age*: int proc greet*(person: Person) = echo "Hello, my name is " & person.name """ * **Don't Do This:** Use global variables or unstructured data when objects or classes would provide better organization and encapsulation. """nim # Bad: Unstructured data var personName: string personAge: int """ * **Why:** Objects and classes provide a powerful way to organize code and data, promoting modularity, reusability, and maintainability. Nim's object system is lightweight and efficient. ### 2.3. Modules * **Do This:** Use modules to group related components and functions. """nim # Good: Module organization (mymodule.nim) module mymodule proc myfunction() = echo "This is my function" """ * **Don't Do This:** Put unrelated components in the same module, or create monolithic modules with too much code. """nim # Bad: Monolithic module module everything proc functionA() proc functionB() proc functionC() """ * **Why:** Modules help to organize code into logical units. Makes dependencies more explicit and improves code discoverability. ### 2.4. Generics * **Do This:** Use generics to create reusable components that work with multiple types. """nim # Good: Generic function proc identity*[T](x: T): T = return x echo identity[int](5) echo identity[string]("hello") """ * **Don't Do This:** Duplicate code for different types when generics can be used. """nim # Bad: Code duplication proc identityInt(x: int): int = return x proc identityString(x: string): string = return x """ * **Why:** Generics reduce code duplication and improve code reusability. They allow you to write components that can work with multiple types without sacrificing type safety. ### 2.5 Concepts (Type Classes) * **Do This:** Leverage concepts to define requirements for generic types. """nim # Good: Using concept to constrain generic type concept Summable[T]: T + T is T proc sum*[T: Summable](a, b: T): T = return a + b echo sum(1, 2) # Works for integers #echo sum("hello", "world") # Compile error as Strings are not Summable """ * **Don't Do This:** Avoid defining generic behavior without proper constraints leading to unexpected errors at runtime. * **Why:** Concepts provide type-safe genericity enabling compile-time checking and better code clarity. They offer a powerful mechanism to define abstract interfaces and enforce type constraints, leading to more robust and maintainable code. ## 3. Design Patterns ### 3.1. Factory Pattern * **Do This:** Use the factory pattern to abstract object creation. """nim # Good: Factory pattern type Animal = interface proc makeSound() {.abstract.} type Dog = object of Animal proc makeSound() {.override.} = echo "Woof!" type Cat = object of Animal proc makeSound() {.override.} = echo "Meow!" proc createAnimal(animalType: string): Animal = case animalType of "dog": return Dog() of "cat": return Cat() else: raise newException(ValueError, "Invalid animal type") let animal = createAnimal("dog") animal.makeSound() """ * **Don't Do This:** Directly instantiate objects in client code, especially when the object creation logic is complex or when the concrete type needs to be hidden. """nim # Bad: Direct instantiation let animal = Dog() animal.makeSound() """ * **Why:** The factory pattern decouples object creation from client code, making the code more flexible and extensible. ### 3.2. Observer Pattern * **Do This:** Use the observer pattern to notify multiple dependent objects when the state of a subject object changes. This can be elegantly accomplished in Nim using closures. """nim # Good: Observer pattern with closures type Subject = object observers: seq[proc (message: string)] proc attach(subject: var Subject, observer: proc (message: string)) = subject.observers.add(observer) proc detach(subject: var Subject, observer: proc (message: string)) = for i, obs in subject.observers: if obs == observer: subject.observers.del(i) break proc notify(subject: Subject, message: string) = for observer in subject.observers: observer(message) var subject: Subject subject.observers = @[] let observer1 = proc (message: string) = echo "Observer 1 received: " & message let observer2 = proc (message: string) = echo "Observer 2 received: " & message subject.attach(observer1) subject.attach(observer2) subject.notify("Hello, observers!") # Output: Observer 1 received: Hello, observers! \n Observer 2 received: Hello, observers! subject.detach(observer2) subject.notify("Second Message") # Output: Observer 1 received: Second Message """ * **Don't Do This:** Directly couple subject objects to dependent objects. This creates tight dependencies and makes it difficult to add or remove observers. * **Why:** Enables a one-to-many dependency relationship without tight coupling. Useful for implementing event handling, reactive systems, and publish-subscribe architectures. **Technology note:** Nim's first class closures make implementing observers straightforward. ### 3.3. Strategy Pattern * **Do This:** Use the strategy pattern to define a family of algorithms, encapsulate each one, and make them interchangeable. """nim # Good: Strategy pattern type SortingStrategy = interface proc sort(data: var seq[int]) {.abstract.} type BubbleSort = object of SortingStrategy proc sort(data: var seq[int]) {.override.} = # Bubble sort implementation type QuickSort = object of SortingStrategy proc sort(data: var seq[int]) {.override.} = # Quicksort implementation type Sorter = object strategy: SortingStrategy proc sort(sorter: Sorter, data: var seq[int]) = sorter.strategy.sort(data) """ * **Don't Do This:** Hardcode algorithms within client code, or use conditional statements to select between algorithms. """nim # Bad: Hardcoded algorithm proc sort(data: var seq[int], algorithm: string) = case algorithm of "bubble": # Bubble sort implementation of "quick": # Quicksort implementation """ * **Why:** Allows you to switch between algorithms at runtime without modifying client code. Promotes flexibility and reusability in algorithm selection. ## 4. Error Handling ### 4.1. Exceptions * **Do This:** Use exceptions to handle exceptional or unexpected situations. Catch exceptions at the appropriate level of abstraction. """nim # Good: Exception handling proc divide(a, b: int): float = if b == 0: raise newException(DivByZeroError, "Division by zero") return float(a) / float(b) try: let result = divide(10, 0) echo result except DivByZeroError: echo "Cannot divide by zero" """ * **Don't Do This:** Ignore exceptions or use them for normal control flow. """nim # Bad: Ignoring exceptions try: let result = divide(10, 0) echo result except: discard # Ignoring the exception """ * **Why:** Exceptions provide a structured way to handle errors and prevent program crashes. Properly handled exceptions improve the robustness and reliability of the system. ### 4.2. Result Types * **Do This:** Use "Result[T, E]" (typically from "std/options") to represent functions that may either return a value of type "T" or an error of type "E". This is the preferred way to handle recoverable errors. """nim import std/options proc parseInt(s: string): Result[int, string] = try: return Ok(parseInt(s)) except ValueError: return Err("Invalid integer format") let result = parseInt("123") case result of Ok(value): echo "Parsed value: ", value of Err(message): echo "Error parsing integer: ", message """ * **Don't Do This:** Rely solely on exceptions for all error scenarios, particularly expected failure conditions. "Result" types provide better compile-time safety and explicit error handling. * **Why:** "Result" types force the caller to handle the potential error case, improving code robustness. They provide a clear and explicit way to represent functions that can fail. They also improve performance as exceptions are costly. ## 5. Documentation ### 5.1. Code Comments * **Do This:** Add comments to explain complex logic, non-obvious code, and the purpose of components. """nim # Good: Meaningful comment proc calculateArea(width, height: int): int = # Calculate the area of a rectangle return width * height """ * **Don't Do This:** Add redundant comments that simply restate the code, or leave code uncommented. """nim # Bad: Redundant comment proc calculateArea(width, height: int): int = # Multiply width and height return width * height """ * **Why:** Code comments are essential for understanding and maintaining code. They provide valuable context for developers who are reading or modifying the code. ### 5.2. API Documentation * **Do This:** Use Nim's documentation generator to create API documentation for components. """nim # Good: API documentation (using doc comments) ## Calculates the area of a rectangle. ## Args: ## width: The width of the rectangle. ## height: The height of the rectangle. ## Returns: ## The area of the rectangle. proc calculateArea*(width, height: int): int = return width * height """ * Run "nim doc yourModule.nim" to generate documentation. Typically "nimble doc" is used for projects. * **Don't Do This:** Neglect to document public APIs, or provide incomplete or inaccurate documentation. * **Why:** API documentation allows others to understand and use your components correctly without having to dive into the implementation details. ## 6. Tooling and Conventions ### 6.1. Compiler Flags * **Do This:** Use appropriate compiler flags to ensure code quality. For example, "-w:error" turns warnings into errors to enforce stricter code standards. Consistent use of a ".nimble" file to specify compiler flags is very useful for project-wide configuration. """nim #Example in .nimble file: requires "nim >= 1.6.0" backend = "cpp" flags="-w:error" """ * **Don't Do This:** Ignore compiler warnings or use inconsistent compiler settings. * **Why:** Using consistent compiler flags improves code quality and helps catch potential errors early in the development process. Turning warnings into errors promotes very high standards. ### 6.2. Code Formatting * **Do This:** Follow a consistent code formatting style. "nimpretty" can automatically format code to a consistent standard. * **Don't Do This:** Use inconsistent formatting, or mix different styles within the same codebase. * **Why:** Consistent code formatting makes the code easier to read and understand. Automation minimizes debates about coding styles. ### 6.3. Linting * **Do This:** Employ a linter (like "nim check") to identify potential code style and semantic issues. Incorporating linting into the build process automates consistent validation. Several linters can be integrated with IDEs providing real-time feedback. * **Don't Do This:** Neglect static analysis, thus allowing common errors and style inconsistencies to go undetected. * **Why:** Linters identify potential issues early, reducing debugging time and improving overall code quality. Automated enforcement reduces the chance of regressions. ## 7. Performance Considerations ### 7.1. Memory Management * **Do this:** Understand Nim’s memory management options (GC, ARC, manual). Choose the best option for the target application (e.g., GC for general use, ARC for deterministic cleanup, manual for full control in performance-critical code). Use "{.gcsafe.}" and "{.noGc.}" pragmas appropriately. """nim #Example of using ARC: {.push gc: arc .} proc processData(data: seq[int]): seq[int] = # Complex processing logic result = newSeq[int](data.len) for i in 0..data.len - 1: result[i] = data[i] * 2 {.pop.} """ * **Don't Do This:** Ignore memory management, assuming the garbage collector (GC) will handle everything optimally. This can lead to memory leaks or performance bottlenecks. * **Why:** Understanding and correctly applying memory management leads to resource-efficient code, minimizing memory usage and avoiding performance degradation. Modern Nim projects increasingly favor ARC for its deterministic cleanup. ### 7.2. Data Structures * **Do This:** Select the most appropriate data structure for the task. Use sequences ("seq") for dynamic arrays, arrays for fixed-size collections, sets ("HashSet") for membership testing, and tables ("Table") or ordered Tables ("OrderedTable") for key-value storage. * **Don't Do This:** Use inappropriate data structures, leading to inefficient operations (e.g., searching a sequence when a set would be more efficient). * **Why:** Efficient data structure usage is crucial for high-performance applications. The appropriate data structure drastically cuts down on processing time. ### 7.3. Iteration * **Do This:** Use iterators for efficient access to collections, especially when transforming data. Consider using parallel iterators ("parallel") for computationally intensive loops. Optimize loops by minimizing computations within loop bodies and using inlining where appropriate. """nim # Example: Efficient iteration and transformation iterator doubledValues(data: seq[int]): int = for item in data: yield item * 2 for value in doubledValues(@[1, 2, 3, 4, 5]): echo value # Output: 2 4 6 8 10 """ * **Don't Do This:** Overlook efficient iteration techniques, performing unnecessary operations within loops. * **Why:** Efficient iteration optimizes data processing and memory access, leading to substantial performance gains, especially in computationally intensive routines. Leverage Nim's powerful iterator features when possible. The "parallel" iterator can unlock significant performance improvement on multi-core hardware. This coding standards document provides guidance for creating high-quality, maintainable, and efficient component designs for Nim. By following these standards, developers can produce code that is easier to understand, test, and reuse.
# State Management Standards for Nim This document outlines the standards for managing state within Nim applications. Effective state management is crucial for building maintainable, scalable, and testable software. It covers different state management approaches applicable to Nim based on the specific needs of an application, including immutability, mutable state with controlled access, and reactive programming. The document emphasizes modern Nim features and preferred patterns. ## 1. Principles of Effective State Management ### 1.1. Immutability **Definition:** An immutable object's state cannot be modified after it is created. Any operation that *appears* to modify it must instead create a new object. **Why it matters:** * **Predictability:** Immutable state simplifies reasoning about code because the value of an object is fixed. * **Thread safety:** Immutable objects are inherently thread-safe. * **Testability:** Easier to test because you don't have to track state changes. * **Caching:** Immutable objects are great for caching, as their validity never changes. **Do This:** * Design data structures as immutable whenever practical. **Don't Do This:** * Unnecessarily mutate state when immutability provides similar functionality. * Share mutable state without careful consideration of concurrency implications. **Example:** """nim type Point = tuple[x, y: int] # tuples are immutable proc movePoint(p: Point, dx, dy: int): Point = ## Returns a new point that is "p" translated by dx, dy. (x: p.x + dx, y: p.y + dy) # creates a new tuple let p1: Point = (x: 1, y: 2) p2 = movePoint(p1, 3, 4) echo p1 # (x: 1, y: 2) echo p2 # (x: 4, y: 6) """ ### 1.2. Controlled Mutability **Definition:** When mutable state is necessary, access to that state should be carefully controlled via well-defined interfaces. **Why it matters:** * **Encapsulation:** Prevents unintended side effects. * **Maintainability:** Makes it easier to understand and modify code. * **Concurrency:** Can be used to manage access to shared resources. **Do This:** * Encapsulate mutable state within objects or modules. * Provide methods or procedures to safely modify state. * Use locks or other synchronization primitives when shared mutable state is accessed by multiple threads. **Don't Do This:** * Expose mutable state directly. * Modify state in unrelated or distant parts of the code. **Example:** """nim import locks type Counter = object value: int lock: Lock proc newCounter(): Counter = Counter(value: 0, lock: newLock()) proc increment(c: var Counter) = acquire(c.lock) try: c.value += 1 finally: release(c.lock) proc getValue(c: Counter): int = acquire(c.lock) try: return c.value finally: release(c.lock) var counter = newCounter() # Example usage within multiple threads import os, threadpool proc threadFunc() = for i in 0..<1000: increment(counter) var threads: array[3, Thread[void]] for i in 0..<threads.len: createThread(threads[i], threadFunc) for i in 0..<threads.len: joinThread(threads[i]) echo "Final value: ", getValue(counter) """ ### 1.3. Explicit Data Flow **Definition:** Make data dependencies clear and predictable. **Why it matters:** * **Readability:** Easier to understand how data is transformed and used. * **Debugging:** Simplifies tracing data through the application. * **Testability:** Makes it easier to isolate units of code for testing. **Do This:** * Pass data explicitly to procedures and functions. * Avoid global variables where possible. * Use clear and descriptive names for variables and parameters. **Don't Do This:** * Rely on implicit state or hidden dependencies. * Use global variables excessively. **Example:** """nim proc calculateArea(width, height: float): float = ## Calculates the area of a rectangle. width * height let w = 10.0 h = 5.0 area = calculateArea(w, h) echo "Area: ", area """ ### 1.4. Reactive Programming (if applicable) **Definition:** A declarative programming paradigm concerned with data streams and the propagation of change. **Why it matters:** * **Responsiveness:** Allows applications to react quickly to changes in data. * **Concurrency:** Can simplify asynchronous programming. * **Declarative style:** Makes it easier to reason about complex interactions. **Do This:** * Consider using reactive libraries like "chronos" or "karax" for UI development. * Use signals or observables to represent data streams. * Use transformations to process and combine data streams. **Don't Do This:** * Overuse reactive programming in simple situations. * Create complex dependency chains that are difficult to debug. **Example using "chronos":** In this example, "chronos" is primarily used for asynchronous operations and its core reactive capabilities not demonstrated. Since Nim doesn't yet have a dominant reactive programming library equivalent to RxJS or similar, a complete reactive example is more involved and would require building custom primitives. """nim import chronos proc main() {.async.} = echo "Starting..." await sleepAsync(1000) # block for 1 second echo "Done!" waitFor main() """ ## 2. Managing Application State ### 2.1. Centralized State **Definition:** Storing all application state in a single, well-defined location, typically an object or module. **Why it matters:** * **Organization:** Provides a clear view of the application's overall state. * **Maintainability:** Simplifies debugging and modification of state. * **Testability:** Makes it easier to set up application-wide tests. **Do This:** * Create a central state object or module. * Define clear accessors and mutators for the state. **Don't Do This:** * Scatter state throughout the application. * Allow direct modification of the state from multiple locations. **Example:** """nim type AppState = object username: string isLoggedIn: bool items: seq[string] var appState: AppState proc initializeState() = appState = AppState(username: "", isLoggedIn: false, items: @[]) proc login(username: string) = appState.username = username appState.isLoggedIn = true proc logout() = appState.username = "" appState.isLoggedIn = false proc addItem(item: string) = appState.items.add(item) proc getItem(index: int): string = if index >= 0 and index < appState.items.len: return appState.items[index] else: return "" initializeState() login("testuser") addItem("item1") echo appState.username echo appState.items """ ### 2.2. State Monads **Definition:** Using a monad to encapsulate and manage state transformations in a functional style. This is more advanced. **Why it matters:** * **Purity:** Makes state transformations explicit and isolated. * **Composability:** Allows complex state transformations to be built from simpler ones. * **Testability:** Each transformation becomes easily testable. **Do This:** * Define a state monad type. * Create helper functions to access and modify the state. * Use "bind" or similar operations to chain state transformations. **Don't Do This:** * Overuse state monads in simple cases. * Create overly complex monad stacks. **Example:** A full State Monad example is a complex topic, better suited to a larger project demonstrating more elaborate transformations. The following is a simplified demonstration of the concepts. Since Nim doesn't have direct syntactic sugar for monads many steps have to be explicitly coded. """nim type State[S, A] = proc (state: S): (A, S) # Represents a state transformation # S is state type, A is returned computation type proc runState[S, A](state: State[S, A], initialState: S): (A, S) = state(initialState) proc unit[S, A](value: A): State[S, A] = proc (state: S): (A, S) = (value, state) proc bind[S, A, B](state: State[S, A], f: proc (a: A): State[S, B]): State[S, B] = proc (s: S): (B, S) = let (a, newState) = state(s) f(a)(newState) # Example: Increment a counter using a state monad type CounterState = int proc getCounter: State[CounterState, CounterState] = proc (state: CounterState): (CounterState, CounterState) = (state, state) # Returns the current state and keeps the state unchanged proc setCounter(newValue: CounterState): State[CounterState, void] = proc (state: CounterState): (void, CounterState) = ((), newValue) # Returns void (unit type) and sets the counter to a new state proc incrementCounter: State[CounterState, void] = bind[CounterState, CounterState, void](getCounter, proc (counter: CounterState): State[CounterState, void] = setCounter(counter + 1) ) # Usage let initialState: CounterState = 0 let ((), finalState) = runState(incrementCounter, initialState) echo "Final counter value: ", finalState """ ### 2.3. Global Variables (Use Sparingly) **Definition:** Variables declared outside of any procedure or object, accessible from anywhere in the code. **Why it matters:** * **Convenience:** Can be easily accessed and modified. * **Performance:** Can avoid the overhead of passing data around. **Don't Do This:** * Global variables should be "const" or "let" whenever possible to prevent accidental modification and improve reasoning. * Use global variables as a primary means of managing application state. * Modify global variables from multiple threads without synchronization. * Global variables with mutable value types are almost always a bad idea, and should be replaced with explicit access-controlled types. **When to use:** * Configuration settings. * Constants that are used throughout the application. * Logging configurations. **Example:** """nim const appName = "MyApplication" version = "1.0.0" var debugMode = false # Only configure here proc log(message: string) = if debugMode: echo appName, " - ", message debugMode = true # Set once during initialization log("Application started") """ ## 3. Data Flow Management ### 3.1. Functional Programming **Definition:** Writing code primarily using pure functions (functions without side effects) and immutable data. **Why it matters:** * **Predictability:** Makes code easier to reason about. * **Testability:** Simplifies unit testing. * **Concurrency:** Eliminates the need for locks and other synchronization primitives. **Do This:** * Use pure functions whenever possible. * Avoid modifying data directly. * Use recursion instead of loops when appropriate. * Leverage Nim's "seq.map", "seq.filter", "seq.foldl" and "seq.foldr" for data processing. **Don't Do This:** * Write functions with side effects that are not clearly documented. * Mutate data unnecessarily. **Example:** """nim proc square(x: int): int = ## Returns the square of x. x * x let numbers = @[1, 2, 3, 4, 5] squares = numbers.map(square) # creates a new sequence with mapped result echo squares # @[1, 4, 9, 16, 25] """ ### 3.2. Message Passing **Definition:** Communicating between different parts of the application by sending and receiving messages. Commonly used for concurrency. **Why it matters:** * **Decoupling:** Reduces dependencies between components. * **Concurrency:** Simplifies parallel programming. * **Resilience:** Improves fault tolerance. **Do This:** * Define clear message formats. * Use channels or queues to send and receive messages. * Handle errors gracefully. **Don't Do This:** * Send large amounts of data in messages. * Block indefinitely waiting for messages. **Example:** """nim import channels import os, threadpool type Message = object text: string var channel: Channel[Message] proc worker() = while true: let message = recv(channel) echo "Received: ", message.text proc main() = channel = newChannel[Message]() var workerThread: Thread[void] createThread(workerThread, worker) send(channel, Message(text: "Hello from main thread!")) send(channel, Message(text: "Another message")) close(channel) # Signal end of communication joinThread(workerThread) main() """ ### 3.3. Data Transformations **Definition:** Emphasizing transformations of data through a series of steps. **Why it matters:** * **Clarity:** Each transformation step is explicit. * **Debuggability:** Easier to track the flow of data. * **Testability:** Individual transformations can be tested in isolation. **Do This:** * Break down complex operations into smaller, well-defined transformations. * Use immutable data structures where appropriate. * Utilize pipelines to chain transformations together. **Don't Do This:** * Create overly complex transformation chains. * Modify data in place during transformations. **Example:** """nim import strutils proc toUpper(s: string): string = ## Converts a string to uppercase. s.toUpperAscii() proc trim(s: string): string = ## Trims whitespace from a string. s.strip() proc addExclamation(s: string): string = ## Adds an exclamation point to a string. s & "!" let message = " hello world " transformedMessage = addExclamation(trim(toUpper(message))) echo transformedMessage """ ## 4. Reactivity and Event Handling ### 4.1. Callbacks **Definition:** Using procedures that are called when specific events occur. **Why it matters:** * **Simple:** Easy to implement for basic event handling. **Do This:** * Use callbacks for simple event handling scenarios. * Consider more advanced mechanisms for complex interactions. **Don't Do This:** * Overuse callbacks leading to callback hell. * Perform long-running operations within callbacks. **Example:** """nim type Button = object onClick: proc () proc createButton(clickAction: proc ()): Button = Button(onClick: clickAction) proc handleClick(button: Button) = button.onClick() proc myCallback() = echo "Button clicked!" let button = createButton(myCallback) handleClick(button) """ ### 4.2. Signals and Slots (Similar to Qt Framework) **Definition:** A mechanism for connecting objects, where a signal emitted by one object triggers a slot (procedure) in another object. Nim does not have built-in support for signals and slots, so this needs to be implemented or a library used **Why it matters:** * **Decoupling:** Reduces dependencies between emitting and receiving objects. * **Flexibility:** Allows multiple slots to be connected to a single signal. * **Extensibility:** Makes it easy to add new event handlers. **Do This:** * Define signal types. * Create procedures to emit signals. * Create procedures as slots to handle signals. * Implement a connection mechanism to link signals and slots. **Don't Do This:** * Create circular dependencies between signals and slots. * Perform long-running operations in slots. **Example (basic implementation):** """nim type Signal = ref object listeners: seq[proc ()] proc newSignal(): Signal = Signal(listeners: @[]) proc connect(signal: Signal, listener: proc ()) = signal.listeners.add(listener) proc emit(signal: Signal) = for listener in signal.listeners: listener() # Example Usage var mySignal = newSignal() proc mySlot() = echo "Signal received!" connect(mySignal, mySlot) # connecting a procedure (mySlot) to a signal emit(mySignal) # Will trigger outputs from any attached slots """ ### 4.3. Message Queues (For Asynchronous Communication) **Definition:** Using a queue to store and process messages asynchronously. *Suitable for cases where responsiveness is key.* **Why it matters:** * **Decoupling:** Allows components to communicate without direct dependencies. * **Scalability:** Enables asynchronous processing of events. * **Resilience:** Improves fault tolerance. **Do This:** * Define message types. * Use a queue to store incoming messages. * Use a separate thread or process to process messages from the queue. **Don't Do This:** * Lose messages due to queue overflow. * Block indefinitely waiting for messages. ## 5. Error Handling ### 5.1. Exceptions **Definition:** Using exceptions to signal errors. **Do This:** * Use exceptions to signal unexpected errors that cannot be handled locally. * Catch exceptions at a higher level to handle errors gracefully. * Use "try...except...finally" blocks to ensure proper cleanup. **Don't Do This:** * Use exceptions for normal control flow. * Ignore exceptions without handling them. * Catch exceptions too broadly. **Example:** """nim proc divide(x, y: int): float = if y == 0: raise newException(CatchableError, "Division by zero") return float(x) / float(y) try: let result = divide(10, 0) echo "Result: ", result except DivisionByZeroError: echo "Error: Cannot divide by zero" finally: echo "Finished division attempt." """ ### 5.2. Result Types **Definition:** Using a type that explicitly represents either a successful value or an error. **Why it matters:** * **Explicit:** Forces the caller to handle the possibility of an error. * **Safe:** Prevents unexpected exceptions. * **Functional:** Fits well with functional programming paradigms. **Do This:** * Define a result type with variants for success and failure. * Return a result type from functions that can fail. * Use case statements or pattern matching to handle results. **Don't Do This:** * Ignore the result of a function that returns a result type. * Throw exceptions after using Result type patterns. **Example:** """nim type Result[T, E] = object case kind: enum {ok, err} of ok: value: T of err: error: E proc divide(x, y: int): Result[float, string] = if y == 0: return Result[float, string](kind: err, error: "Division by zero") else: return Result[float, string](kind: ok, value: float(x) / float(y)) let result = divide(10, 0) case result.kind of ok: echo "Result: ", result.value of err: echo "Error: ", result.error """ ## 6. Tooling and Libraries. ### 6.1. Testing Frameworks * Use a testing framework such as "unittest" to write unit tests. * Write tests for all critical code paths. * Use mocking libraries to isolate units of code for testing. ### 6.2. Debugging Tools * Use a debugger to step through code and inspect variables. The Nim compiler has GDB integration. * Use logging to record information about the execution of the application. ## 7. Security Considerations ### 7.1. Input Validation * Validate all input data to prevent security vulnerabilities. * Use appropriate encoding and escaping techniques to prevent injection attacks. ### 7.2. Data Encryption * Encrypt sensitive data both in transit and at rest. * Use strong encryption algorithms and key management practices. ### 7.3. Access Control * Implement appropriate access controls to protect sensitive data and resources. * Use the principle of least privilege. ## 8. Conclusion Effective State Management is critical for the development of robust and maintainable Nim applications. By adhering to these standards and guidelines, developers can create well-structured, secure, and performant code. The correct approach depends significantly on the complexity of the specific application being developed. The principles outlined above should, however, always be a consideration.
# Testing Methodologies Standards for Nim This document outlines the standards for testing methodologies in Nim projects. Consistent testing practices are crucial for ensuring code quality, reliability, and maintainability. These guidelines cover unit, integration, and end-to-end testing, with an emphasis on modern approaches and leveraging the Nim ecosystem. ## 1. General Testing Principles ### 1.1. Writing Testable Code * **Do This**: Design code with testability in mind from the start. Use dependency injection, modular design, and interfaces to promote separation of concerns. * **Why**: Testable code is easier to isolate and verify. Clean architecture simplifies the testing process and reduces the likelihood of brittle tests. """nim # Good: Using interfaces and dependency injection type DataFetcher = interface proc fetchData(): string type RealDataFetcher = object of DataFetcher # Implementation details MockDataFetcher = object of DataFetcher data: string proc fetchData(): string = data proc processData(fetcher: DataFetcher): string = let data = fetcher.fetchData() # Process the data # In tests: let mockFetcher = MockDataFetcher(data: "Test Data") let result = processData(mockFetcher) """ * **Don't Do This**: Write monolithic functions or tightly coupled classes without considering how they will be tested. * **Why**: Such code is difficult to isolate, leading to complex and unreliable tests. ### 1.2. Test-Driven Development (TDD) * **Do This**: Consider adopting TDD by writing tests before implementing the actual code. This forces you to think about the desired behavior and API design ahead of time. * **Why**: TDD leads to cleaner code, better architecture, and higher test coverage. ### 1.3. Test Naming Conventions * **Do This**: Use clear and descriptive names for test functions and files that clearly indicate the tested functionality. A common convention is "test_module_function_scenario". * **Why**: Good naming improves readability and makes it easier to understand the purpose of each test. """nim # Good: Descriptive test name test "strutils.parseInt - valid integer string returns int": check parseInt("123") == 123 test "strutils.parseInt - invalid integer string raises ValueError": expect ValueError: discard parseInt("abc") """ * **Don't Do This**: Use vague or cryptic test names that do not clearly indicate what is being tested. ### 1.4. Test Organization * **Do This**: Structure tests in a way that mirrors the project's file structure. For instance, if your module is "my_module.nim", place the tests in "tests/test_my_module.nim". * **Why**: This consistency makes it easier to find tests and maintain the testing suite. * **Do This**: Use separate source code folders to keep test and app code separate. * **Why**: This avoids deploying the tests into production code ### 1.5. First Principles in Testing * **Do This**: Follow the FIRST principles of testing: * **F**ast: Tests should run quickly, enabling frequent execution. * **I**ndependent: Tests should not depend on each other. * **R**epeatable: Tests should produce the same results every time. * **S**elf-validating: Tests should automatically determine if they passed or failed. * **T**horough: Test all relevant scenarios and edge cases. * **Why**: Adhering to these principles ensures that tests are reliable and provide valuable feedback. ## 2. Unit Testing ### 2.1. Purpose of Unit Tests * **Do This**: Use unit tests to verify the behavior of individual functions, procedures, methods, or classes in isolation. * **Why**: Unit tests are the foundation of a robust testing strategy. They help catch bugs early and prevent regressions. ### 2.2. Using the "unittest" Module * **Do This**: Leverage Nim's built-in "unittest" module for writing and running unit tests. Use "check", "expect", and "suite" blocks. """nim import unittest suite "MyModule Tests": test "MyFunction - Positive Case": check myFunction(1, 2) == 3 test "MyFunction - Negative Case": check myFunction(-1, 2) == 1 test "MyFunction - Zero Case": check myFunction(0, 0) == 0 test "MyFunction - Exception Handling": expect ValueError: discard myFunction(1, -1) """ * **Don't Do This**: Reinvent the wheel by writing your own testing framework from scratch. The "unittest" module provides all the necessary features. ### 2.3. Mocking and Stubbing * **Do This**: Use mocking and stubbing to isolate the unit under test from its dependencies. * **Why**: Mocking allows you to control the behavior of external components, enabling you to test specific scenarios without relying on actual dependencies. * **How**: Typically this is done using interfaces or by creating simple mock objects: """nim import unittest type DataFetcher = interface proc fetchData(): string type RealDataFetcher = object of DataFetcher # Implementation details MockDataFetcher = object of DataFetcher data: string proc fetchData(): string = data proc processData(fetcher: DataFetcher): string = let data = fetcher.fetchData() return data suite "Data Processing Tests": test "processData - uses mock data": let mockFetcher = MockDataFetcher(data: "Test Data") let result = processData(mockFetcher) check result == "Test Data" """ * **Don't Do This**: Directly depend on real external services or databases in your unit tests, as it makes them slow, unreliable, and difficult to set up. ### 2.4. Assertion Strategies * **Do This**: Use appropriate assertion methods (e.g., "check", "expect", "assert") to verify different aspects of the code's behavior. * **Why**: Using specific assertion methods improves test clarity and provides more informative error messages. * The "check" block verifies that a condition is true. * The "expect" block verifies that a specific exception is raised. * The "assert" statement checks a condition and terminates the program if it fails. Use with caution in tests. """nim import unittest proc divide(a, b: int): float = if b == 0: raise newException(ValueError, "Cannot divide by zero") return float(a) / float(b) suite "Division Tests": test "divide - positive numbers": check divide(10, 2) == 5.0 test "divide - divide by zero raises error": expect ValueError: discard divide(10, 0) test "equality test": var x: int = 5 y: int = 5 check x == y test "inequality test": var x: int = 5 y: int = 6 check x != y """ * **Don't Do This**: Use overly generic assertions or rely on side effects to determine test results. ### 2.5. Test Coverage * **Do This**: Aim for high test coverage, but prioritize testing critical paths and edge cases. Leverage tools like "koch coverage" to measure coverage. * **Why**: High test coverage reduces the risk of undetected bugs, especially regressions. * **How**: To generate code coverage reports, use the "--coverage" compilation flag and then run "koch coverage". """bash nim compile --coverage my_module.nim ./my_module # Run the program or tests koch coverage """ * **Don't Do This**: Blindly aim for 100% test coverage without considering the value of each test. Prioritize testing complex logic and critical functionalities. ## 3. Integration Testing ### 3.1. Purpose of Integration Tests * **Do This**: Use integration tests to verify the interaction between different modules or components of the system. * **Why**: Integration tests ensure that the various parts of the application work correctly together. ### 3.2. Testing Strategies * **Do This**: Test the interaction between modules. Consider using a layered approach where you test interactions within a layer, and then interactions between layers. * **Why**: Integration tests confirm that the modules are compatible and properly communicate. """nim # Example: Testing interaction between a data layer and a business logic layer proc fetchDataFromDatabase(): seq[string] = # Simulate fetching data from a database return @["Data 1", "Data 2"] proc processData(data: seq[string]): seq[string] = # Simulate processing the data return data.mapIt(it & " Processed") proc getProcessedData(): seq[string] = let data = fetchDataFromDatabase() return processData(data) import unittest suite "Integration Tests": test "getProcessedData - fetches and processes data correctly": let result = getProcessedData() check result == @["Data 1 Processed", "Data 2 Processed"] """ * **Don't Do This**: Skip integration tests and assume that unit tests are sufficient. Integration tests can uncover issues that unit tests miss. ### 3.3. Database Testing * **Do This**: Use a test database (e.g., an in-memory SQLite database) for integration tests that involve database interactions. * **Why**: Test databases provide a clean and isolated environment for testing database-dependent code. """nim # Example using SQLite import sqlite3, unittest var db: Database suite "Database Integration Tests": setup: db = open(":memory:") db.exec "CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)" teardown: db.close() test "insert and select user": db.exec "INSERT INTO users (name) VALUES ('Test User')" let result = db.execAndCollect "SELECT name FROM users WHERE id = 1" check result[0][0] == "Test User" """ * **Don't Do This**: Run integration tests against a production database, as this can lead to data corruption or unintended side effects. ### 3.4. API Testing * **Do This**: Write integration tests to verify the behavior of REST APIs or other external services. * **Why**: API tests ensure that your application can correctly interact with external systems. """nim # Example using httpclient import httpclient, unittest, json proc fetchUserData(userId: int): JsonNode = let client = newHttpClient() let response = client.get("https://jsonplaceholder.typicode.com/users/" & $userId) if response.status == Http200: return parseJson(response.body) else: raise newException(Exception, "API request failed") suite "API Integration Tests": test "fetchUserData - returns valid user data": let userData = fetchUserData(1) check userData["id"].getInt == 1 check userData["name"].getStr != "" """ * **Don't Do This**: Hardcode API keys or sensitive information in your integration tests. Use environment variables or configuration files to manage credentials. ## 4. End-to-End (E2E) Testing ### 4.1. Purpose of E2E Tests * **Do This**: Use E2E tests to verify the complete workflow of the application from the user's perspective. Simulate real user interactions. * **Why**: E2E tests ensure that all parts of the system work together seamlessly and that the application meets the user's requirements. ### 4.2. Testing Strategies * **Do This**: Automate user interactions using tools like Selenium or Playwright. Focus on testing critical user journeys. * **Why**: E2E tests provide the highest level of confidence that the application works as expected. * **Don't Do This**: Rely solely on manual testing for E2E scenarios. Automation provides faster feedback and reduces the risk of human error. ### 4.3. Setup and Teardown * **Do This**: Create a clean testing environment before each E2E test and restore the environment to its original state after the test. * **Why**: This prevents tests from interfering with each other and ensures consistent results. ### 4.4. Example The example below uses a fictional cli-app "simpleapp" to make a point about end-to-end testing. Real E2E testing would involve tools like Selenium and the app being deployed, however, it helps to illustrate testing the overall application flow. """nim import unittest, os, strutils, osproc suite "End-to-End Tests": test "simpleapp - valid input produces correct output": let result = execCmdEx("./simpleapp", input = "10 20") check result.exitCode == 0 check result.output.strip() == "The sum is: 30" test "simpleapp - invalid input produces error message": let result = execCmdEx("./simpleapp", input = "abc 20") check result.exitCode != 0 check result.output.contains("invalid integer") """ ### 4.5. Reporting * **Do This**: Generate comprehensive test reports that include information about test execution time, pass/fail status, and error messages. * **Why**: Detailed reports help identify and troubleshoot issues quickly. ## 5. Continuous Integration (CI) ### 5.1. Integrating Tests into CI * **Do This**: Integrate your tests into a CI/CD pipeline to automatically run tests whenever code is committed or merged. * **Why**: CI provides early feedback on code quality and prevents broken code from being merged into the main branch. ### 5.2. Common Platforms * **Do This**: Use popular CI platforms like GitHub Actions, GitLab CI, or Jenkins to automate your testing process. * **Why**: These platforms provide the infrastructure and tools needed to run tests efficiently. ### 5.3. Example GitHub Actions """yaml # .github/workflows/main.yml name: CI on: push: branches: [ "main" ] pull_request: branches: [ "main" ] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Setup Nim uses: actions/setup-nim@v1 with: nim-version: '1.6' # Or the latest version - name: Install dependencies run: | nimble install -y unittest - name: Compile run: | nim compile --verbosity:2 simpleapp.nim nim c -r unittest_reporter.nim - name: Run tests run: | ./unittest_reporter tests/test_simpleapp.nim """ ## 6. Performance Testing ### 6.1. Importance of Performance Testing * **Do This**: Implement performance tests to ensure the efficiency and scalability of your Nim code, especially in performance-critical applications. * **Why**: Nim is often used for performance-sensitive tasks; identifying and addressing bottlenecks is essential. ### 6.2. Profiling Tools * **Do This**: Use profiling tools like "gcstats" or specialized profilers to identify performance bottlenecks. * **Why**: Profilers provide insights into CPU usage, memory allocation, and other performance metrics. ### 6.3. Benchmarking * **Do This**: Use the "benchmarks" suite in the standard library to perform benchmark tests. * **Why**: Benchmarking helps compare the performance of different implementations and track performance improvements over time. """nim import benchmarks proc myFunction(n: int): int = var result = 0 for i in 1..n: result += i return result benchmark "myFunction with n=1000": myFunction(1000) benchmark "myFunction with n=100000": myFunction(100000) """ ### 6.4. Load Testing * **Do This**: Simulate concurrent users to assess the application's behavior under load. * **Why**: Load testing helps identify scalability issues and ensures that the application can handle expected traffic. """nim # Example import httpclient, strutils, os, locks, threadpool const numRequests = 100 numThreads = 10 var successCount = 0 failureCount = 0 lock: Lock proc makeRequest() {.thread.} = let client = newHttpClient() try: let response = client.get("http://localhost:8080") # Replace with your URL if response.status == Http200: acquire(lock) successCount += 1 release(lock) else: acquire(lock) failureCount += 1 release(lock) echo "Request failed with status: ", response.status except Exception as e: acquire(lock) failureCount += 1 release(lock) echo "Exception: ", e.msg proc main() = createLock(lock) var pool: ThreadPool[void] pool.create(numThreads) for i in 1..numRequests: pool.post(makeRequest) pool.join() destroyLock(lock) echo "Successes: ", successCount echo "Failures: ", failureCount main() """ ## 7. Security Testing ### 7.1. Importance of Security Testing * **Do This**: Incorporate security testing into the development process to identify and address vulnerabilities. * **Why**: Security testing helps prevent security breaches and protects sensitive data. ### 7.2. Common Vulnerabilities * **Do This**: Test for common vulnerabilities such as: * SQL injection * Cross-site scripting (XSS) * Cross-site request forgery (CSRF) * Authentication and authorization issues * **Why**: Addressing these vulnerabilities is crucial for building secure applications. ### 7.3. Tools * **Do This**: Use static analysis tools like "semgrep" or dynamic analysis tools to identify potential security flaws. * **Why**: These tools can help automate the process of finding vulnerabilities. ### 7.4. Fuzzing * **Do This**: Use fuzzing to test the application's robustness against invalid or unexpected inputs. * **Why**: Fuzzing can uncover hidden bugs and security vulnerabilities. ## 8. Code Review ### 8.1. Purpose of Code Review * **Do This**: Conduct code reviews to ensure that tests are well-written, comprehensive, and maintainable. * **Why**: Code reviews improve code quality and help share knowledge within the team. ### 8.2. Review Checklist * **Do This**: Use a checklist to guide the code review process. The checklist should include items such as: * Are tests clear and descriptive? * Do tests cover all relevant scenarios? * Are mocks and stubs used appropriately? * Are tests fast and reliable? * Is test coverage adequate? * **Why**: A checklist ensures that code reviews are thorough and consistent. ## 9. Documentation ### 9.1. Documenting Tests * **Do This**: Document the purpose and behavior of tests, especially for complex or critical functionalities. * **Why**: Documentation makes it easier to understand and maintain the testing suite. ### 9.2. Example Documentation """nim # Tests the behavior of the "processData" function. # # This function takes a sequence of strings as input and applies a transformation # to each string. The tests verify that the transformation is applied correctly # and that the function handles different input scenarios, including empty sequences # and sequences with special characters. suite "Data Processing Tests": test "processData - processes data correctly": # ... """ ## 10. Using the testutils module Always make sure that you use the correct module for you test needs. For example, the testutils module has many helpful functions for comparing floating point numbers by accounting for rounding errors. """nim import unittest import testutils suite "floating_point_tests": test "comparing close floats": let x = 0.1 + 0.2 y = 0.3 check isNear(x, y) check isNear(1e-10, 0) test "comparing floats with an epsilon": let x = 1.0 y = 1.0 + 1e-7 check isNear(x, y, eps = 1e-6) # Compare with a custom epsilon """ These guidelines provide a comprehensive foundation for testing in Nim. By adhering to these standards, development teams can create high-quality, reliable, and maintainable applications. Remember that adapting these standards to the specific needs of each project is essential for achieving optimal results.