# Performance Optimization Standards for Ollama
This document outlines performance optimization standards for developing within the Ollama ecosystem. It provides guidelines and best practices to improve application speed, responsiveness, and resource utilization when building, deploying, and using Ollama models. This is intended to be a living document, reflecting the evolving landscape of Ollama and its related technologies.
## 1. Architecture and Model Selection
### 1.1. Optimize Model Size and Precision
**Standard:** Select the smallest model required to meet accuracy requirements. Prioritize lower precision models (e.g., "Q4_0" or "Q5_K_M") when possible.
**Why:** Larger models consume more memory and processing power, impacting latency and throughput. Lower precision models can drastically reduce memory footprint and accelerate calculations, at the expense of potentially lower accuracy.
**Do This:** Evaluate model accuracy using a representative dataset across varying quantization levels.
**Don't Do This:** Blindly default to the largest or highest precision model available.
**Example:**
"""bash
# Consider quantized models if acceptable accuracy can be maintained for a specific use-case scenario
ollama run llama2:7b-chat-Q4_0 "Summarize: " # use a lower precision model Q4_0
ollama run llama2:7b-chat "Summarize: " # high precision
"""
**Anti-pattern:**
Assuming that higher precision always leads to significantly better results without benchmarking. The increased computational cost might outweigh the marginal gains in accuracy for certain tasks.
### 1.2. Strategic Model Caching
**Standard:** Utilize Ollama's caching mechanisms effectively to minimize redundant model loading. Ensure enough disk space is available for the cache directory. Also consider implementing application-level caching for generated responses.
**Why:** Loading models from disk is a significant bottleneck. Caching allows for faster reuse and reduces initialization overhead.
**Do This:** Ensure the Ollama service is configured correctly with sufficient disk space for caching. Implement application-level caching (e.g., using Redis or Memcached) for common prompts and responses.
**Don't Do This:** Rely solely on Ollama's internal caching without considering application-level caching for frequently accessed data. Manually delete or interfere with Ollama's cache directory.
**Example:**
*Ollama configuration:* Ensure the "OLLAMA_MODELS" environment variable points to a directory with sufficient storage space. Consider mounting a larger disk volume if necessary.
"""python
# Application-level caching - basic example using functools.lru_cache
import functools
from ollama import generate
@functools.lru_cache(maxsize=128) # Adjust maxsize as needed
def get_ollama_response(prompt, model="llama2"):
response = generate(model=model, prompt=prompt)
return response['response']
# Use the cached function
result1 = get_ollama_response("Summarize this short text: AI is powerful")
result2 = get_ollama_response("Summarize this short text: AI is powerful") # Second call will be from cache
"""
**Anti-pattern:** Not implementing any caching mechanisms, leading to repeatedly loading the same model and generating identical responses for the same prompt.
### 1.3. Asynchronous Operations
**Standard:** Utilize asynchronous operations where possible to prevent blocking the main thread, improving responsiveness, especially for long-running tasks.
**Why:** Ollama interactions (model loading, inference) can sometimes take a while. Asynchronous execution allows other tasks to proceed concurrently, maintaining a responsive user experience.
**Do This:** Use Python's "asyncio" or similar libraries to handle Ollama requests in a non-blocking manner.
**Don't Do This:** Perform Ollama calls synchronously within the main thread, causing UI freezes or blocking other critical operations.
**Example:**
"""python
import asyncio
import ollama
async def generate_async(model, prompt):
response = await asyncio.to_thread(ollama.generate, model=model, prompt=prompt, stream=False)
return response['response']
async def main():
task1 = asyncio.create_task(generate_async(model="llama2", prompt="What is 2 + 2?"))
task2 = asyncio.create_task(generate_async(model="llama2", prompt="What is the capital of France?"))
results = await asyncio.gather(task1, task2)
print(f"Result 1: {results[0]}")
print(f"Result 2: {results[1]}")
if __name__ == "__main__":
asyncio.run(main())
"""
**Anti-pattern:** Synchronously calling Ollama functions within a web server's request handler, leading to slow response times and potentially overwhelming the server. This is especially impactful in high-concurrency environments.
## 2. Code Optimization
### 2.1. Efficient Prompt Engineering
**Standard:** Craft prompts that are clear, concise, and tailored to the specific task. Avoid unnecessary complexity and redundancy.
**Why:** Well-designed prompts improve accuracy and reduce the amount of processing required by the model. Reducing the token count in your prompts drastically reduces computation time and memory usage.
**Do This:** Experiment with different prompt formulations and evaluate their impact on both accuracy and inference speed.
**Don't Do This:** Use overly verbose or ambiguous prompts that can confuse the model or lead to irrelevant computations.
**Example:**
Instead of: "I have a document here. It's about the history of the Roman Empire. Can you summarize it for me in a concise way that is easy to understand and focuses on the key events?"
Use: "Summarize the key events of the Roman Empire."
**Anti-pattern:** Providing the model with lengthy instructions or background context that is not essential to the task, increasing the overall token length and processing time.
### 2.2. Streaming Responses
**Standard:** Enable streaming responses for long-running generation tasks. This allows for partial results to be displayed to the user as they are generated, improving perceived responsiveness.
**Why:** Streaming provides a better user experience by allowing users to see progress in real-time, rather than waiting for the entire response to be generated.
**Do This:** Utilize the "stream=True" parameter in the "ollama.generate" function. Handle the streamed output appropriately in your application.
**Don't Do This:** Buffer the entire response before displaying it to the user, negating the benefits of streaming.
**Example:**
"""python
import ollama
for chunk in ollama.generate(model='llama2', prompt='Tell me a story.', stream=True):
print(chunk['response'], end='', flush=True) # Print chunks as they come
print('\n')
"""
**Anti-pattern:** Ignoring the "stream" option and waiting for the entire response to be generated on the server side before sending it to the client. This results in a degraded user experience.
### 2.3. Batch Processing
**Standard:** If possible, batch multiple independent prompts together into a single request to the Ollama API.
**Why:** Batching can significantly improve throughput by reducing the overhead associated with individual API calls.
**Do This:** Structure your application to group similar tasks into batches and submit them to Ollama simultaneously. Consider carefully the tradeoffs around batch size and latency, as larger batches may increase latency for individual requests.
**Don't Do This:** Serially process individual prompts when a batched approach would be more efficient.
**Example:** (Requires adapting the "ollama" library – currently does not natively support batching. Illustrative concept only)
"""python
# Hypothetical Batching example (needs modification of ollama library)
# Not directly supported by current ollama library
prompts= ["Tell me a story about a cat", "What is 2 + 2?", "Summarize the key events of WWII"]
# Assuming modification to ollama to accept batches of prompts
responses=ollama.generate(model='llama2, prompts=prompts, stream=False)
for i, response in enumerate(responses):
print (f"Response {i+1}:{response['response']}")
"""
**Anti-pattern:** Making individual API calls for each prompt, even when these prompts could be processed in parallel or batched together.
### 2.4. Optimize Data Structures and Algorithms
**Standard:** Use appropriate data structures and algorithms to minimize memory usage and processing time in your application code that interacts with Ollama.
**Why:** Efficient code is crucial for minimizing overhead and maximizing performance, especially when dealing with large datasets or complex computations.
**Do This:**
* Use Python's built-in data structures (lists, dictionaries, sets) judiciously.
* Consider using libraries like NumPy or Pandas for numerical computations or data manipulation.
* Profile your code to identify performance bottlenecks and optimize accordingly.
**Don't Do This:**
* Use inefficient algorithms or data structures that lead to unnecessary memory consumption or processing time.
* Prematurely optimize code without profiling to identify actual performance bottlenecks.
**Example:**
"""python
import time
import numpy as np
# Inefficient approach (creating large lists and iterating)
def inefficient_sum(n):
numbers = []
for i in range(n):
numbers.append(i)
total = 0
for num in numbers:
total += num
return total
# Efficient approach (using NumPy array and vectorized operations)
def efficient_sum(n):
numbers = np.arange(n) # Create a NumPy array
total = np.sum(numbers) # Use NumPy's sum function
return total
n = 1000000 # Large n
start_time = time.time()
inefficient_sum(n)
end_time = time.time()
print(f"Inefficient approach time: {end_time - start_time:.4f} seconds")
start_time = time.time()
efficient_sum(n)
end_time = time.time()
print(f"Efficient approach time: {end_time - start_time:.4f} seconds")
"""
**Anti-pattern:** Loading an entire dataset into memory when only a subset of the data is needed for processing.
## 3. Hardware and Infrastructure
### 3.1. GPU Acceleration
**Standard:** Utilize GPU acceleration for faster inference, especially when running large models or processing a high volume of requests.
**Why:** GPUs are designed for parallel processing, making them significantly faster than CPUs for the matrix multiplication operations involved in neural network inference.
**Do This:** Ensure Ollama is configured to use a compatible GPU. Install the necessary drivers and libraries (e.g., CUDA or ROCm).
**Don't Do This:** Rely solely on CPU inference for performance-critical applications.
**Example:**
* **Check GPU availability:** Ollama automatically detects available GPUs. Verify correct detection using system tools (e.g., "nvidia-smi").
* **Ensure correct CUDA installation (Nvidia):** Follow the official Nvidia documentation for installing CUDA drivers compatible with your GPU and operating system.
### 3.2. Resource Allocation
**Standard:** Allocate sufficient resources (CPU, memory, GPU) to the Ollama service to prevent resource contention and ensure optimal performance.
**Why:** Insufficient resources can lead to slow inference times, timeouts, and application instability.
**Do This:** Monitor resource usage (CPU, memory, GPU) and adjust resource allocation accordingly. Consider using container orchestration tools (e.g., Kubernetes) to manage resource allocation dynamically.
**Don't Do This:** Under-provision resources, leading to performance bottlenecks. Over-provision resources unnecessarily, wasting resources.
**Example:**
* **Monitor resource usage:** Use tools like "top", "htop", "nvidia-smi", or dedicated monitoring solutions to track resource consumption.
* **Container resource limits (Docker Compose):**
"""yaml
version: "3.8"
services:
ollama:
image: ollama/ollama
ports:
- "11434:11434"
deploy:
resources:
limits:
cpus: '4'
memory: 8G
reservations:
cpus: '2'
memory: 4G
"""
### 3.3. Network Optimization
**Standard:** Minimize network latency between the application and the Ollama service.
**Why:** Network latency can significantly impact the overall response time, especially for applications that make frequent calls to the Ollama API.
**Do This:** Deploy the application and the Ollama service in the same data center or network region. Use efficient network protocols (e.g., HTTP/2 or gRPC). Consider using a content delivery network (CDN) to cache Ollama responses closer to the user.
**Don't Do This:** Deploy the application and the Ollama service across geographically distant regions.
**Example:**
* **Co-locate services:** Deploy both the application and Ollama on the same virtual network or within the same Kubernetes cluster to minimize network hops and latency.
* **Use HTTP Keep-Alive:** Enable HTTP Keep-Alive to reuse existing TCP connections for multiple requests.
## 4. Monitoring and Profiling
### 4.1. Comprehensive Monitoring
**Standard:** Implement comprehensive monitoring to track key performance indicators (KPIs) such as inference time, throughput, error rates, and resource utilization.
**Why:** Monitoring is essential for identifying performance bottlenecks, detecting anomalies, and ensuring the stability of the application.
**Do This:** Use monitoring tools like Prometheus, Grafana, or cloud-based monitoring services to collect and visualize performance data. Set up alerts to notify you of potential issues.
**Don't Do This:** Operate the application without any monitoring in place.
**Example:**
* **Prometheus metrics:** Expose Ollama metrics in Prometheus format. Configure Prometheus to scrape these metrics periodically.
* **Grafana dashboards:** Create Grafana dashboards to visualize KPIs such as inference time, throughput, and resource utilization.
### 4.2. Profiling and Benchmarking
**Standard:** Profile the application code to identify performance bottlenecks and optimize accordingly. Benchmark different models and configurations to determine the optimal settings for your specific use case.
**Why:** Profiling and benchmarking provide valuable insights into the performance characteristics of the application and allow you to make informed decisions about optimization.
**Do This:** Use profiling tools like Python's "cProfile" or dedicated performance analysis tools to identify slow code paths. Benchmark different models and configurations using representative datasets.
**Don't Do This:** Rely on guesswork or intuition when optimizing performance.
**Example:**
"""python
import cProfile
import time
def test_ollama_performance():
import ollama
start_time = time.time()
for i in range(10):
ollama.generate(model="llama2", prompt="Tell me a short story")
end_time= time.time()
print(f"Total time= {end_time-start_time}")
cProfile.run('test_ollama_performance()')
"""
**Output includes a detailed breakdown of which parts of your function is taking up the most time.**
**Summary:** These standards are designed to provide a systematic approach to address performance within the Ollama ecosystem in a comprehensive manner that covers key areas: Architectural Model Selection, Code Optimization, Hardware and Infrastructure, and Monitoring/Profiling.
danielsogl
Created Mar 6, 2025
This guide explains how to effectively use .clinerules with Cline, the AI-powered coding assistant.
The .clinerules file is a powerful configuration file that helps Cline understand your project's requirements, coding standards, and constraints. When placed in your project's root directory, it automatically guides Cline's behavior and ensures consistency across your codebase.
Place the .clinerules file in your project's root directory. Cline automatically detects and follows these rules for all files within the project.
# Project Overview project: name: 'Your Project Name' description: 'Brief project description' stack: - technology: 'Framework/Language' version: 'X.Y.Z' - technology: 'Database' version: 'X.Y.Z'
# Code Standards standards: style: - 'Use consistent indentation (2 spaces)' - 'Follow language-specific naming conventions' documentation: - 'Include JSDoc comments for all functions' - 'Maintain up-to-date README files' testing: - 'Write unit tests for all new features' - 'Maintain minimum 80% code coverage'
# Security Guidelines security: authentication: - 'Implement proper token validation' - 'Use environment variables for secrets' dataProtection: - 'Sanitize all user inputs' - 'Implement proper error handling'
Be Specific
Maintain Organization
Regular Updates
# Common Patterns Example patterns: components: - pattern: 'Use functional components by default' - pattern: 'Implement error boundaries for component trees' stateManagement: - pattern: 'Use React Query for server state' - pattern: 'Implement proper loading states'
Commit the Rules
.clinerules in version controlTeam Collaboration
Rules Not Being Applied
Conflicting Rules
Performance Considerations
# Basic .clinerules Example project: name: 'Web Application' type: 'Next.js Frontend' standards: - 'Use TypeScript for all new code' - 'Follow React best practices' - 'Implement proper error handling' testing: unit: - 'Jest for unit tests' - 'React Testing Library for components' e2e: - 'Cypress for end-to-end testing' documentation: required: - 'README.md in each major directory' - 'JSDoc comments for public APIs' - 'Changelog updates for all changes'
# Advanced .clinerules Example project: name: 'Enterprise Application' compliance: - 'GDPR requirements' - 'WCAG 2.1 AA accessibility' architecture: patterns: - 'Clean Architecture principles' - 'Domain-Driven Design concepts' security: requirements: - 'OAuth 2.0 authentication' - 'Rate limiting on all APIs' - 'Input validation with Zod'
# Testing Methodologies Standards for Ollama This document outlines the testing methodologies standards for contributing to the Ollama project. Adhering to these standards ensures code quality, maintainability, reliability, and security. These guidelines should be used by developers and AI coding assistants alike. ## 1. Overview of Testing in Ollama Testing in Ollama is crucial for ensuring the stability and reliability of the platform. Given Ollama's role in serving and managing large language models, robust testing is essential to guarantee correct model serving, efficient resource utilization, and secure operation. This document outlines unit, integration, and end-to-end testing approaches tailored for Ollama. ### 1.1. Test Pyramid We follow the test pyramid concept, emphasizing a large number of unit tests, a reasonable amount of integration tests, and fewer end-to-end tests. This balance helps provide comprehensive coverage while minimizing test execution time and maximizing isolation. ### 1.2. Test Driven Development (TDD) While not strictly enforced, Test Driven Development (TDD) is encouraged, especially for new features. Writing tests before implementation helps clarify requirements and ensures that code is testable from the start. ## 2. Unit Testing Standards Unit tests should focus on testing individual components or functions in isolation. They should be fast, reliable, and easy to maintain. In Ollama, these tests are particularly important for the core components like model loading, inference engines, and API handlers. ### 2.1. Guidelines * **Do This**: Aim for high code coverage (ideally > 80%) for all core components. * **Do This**: Mock external dependencies (e.g., file system access, network calls) to isolate the unit under test. * **Do This**: Write clear and descriptive test names that accurately reflect the behavior being tested. * **Do This**: Use assertion libraries to verify expected outcomes. * **Don't Do This**: Perform I/O operations or rely on external services in unit tests. * **Don't Do This**: Write overly complex tests that are difficult to understand or maintain. * **Don't Do This**: Neglect edge cases and boundary conditions. * **Why**: Unit tests provide the fastest feedback loop and are essential for catching regressions early. ### 2.2. Specific Implementations & Examples Assume Ollama uses Go as its primary language. """go // Example: Unit test for a function that loads a model configuration package model import ( "testing" "os" ) func TestLoadModelConfig(t *testing.T) { // Create a temporary file with a sample model config tempFile, err := os.CreateTemp("", "model_config.yaml") if err != nil { t.Fatalf("Failed to create temporary file: %v", err) } defer os.Remove(tempFile.Name()) // Clean up after the test defer tempFile.Close() _, err = tempFile.WriteString(" name: my_model format: llama params: temperature: 0.7 top_p: 0.9 ") if err != nil { t.Fatalf("Failed to write to temporary file: %v", err) } //reset file pointer tempFile.Seek(0,0) // Call the function being tested (loadModelConfig) config, err := loadModelConfig(tempFile.Name()) // Assuming loadModelConfig takes the filename // Assert that there is no error and the config is loaded correctly if err != nil { t.Fatalf("Failed to load model config: %v", err) } if config.Name != "my_model" { t.Errorf("Expected model name 'my_model', got '%s'", config.Name) } if config.Format != "llama"{ t.Errorf("Expected model format 'llama', got '%s'", config.Format) } if config.Params["temperature"] != 0.7 { t.Errorf("Expected temperature 0.7, got %f", config.Params["temperature"]) } } //Example mocking function func loadModelConfig(filename string) (*ModelConfig, error) { //implementation accessing file system return &ModelConfig{ Name: "test_model", Format: "test_format", Params: map[string]interface{}{ "temperature": 0.5, }, }, nil } type ModelConfig struct { Name string Format string Params map[string]interface{} } """ **Explanation:** * The test creates a temporary file containing a sample model configuration. * It then calls the "loadModelConfig" and asserts that it returns no error and that the loaded configuration has the expected values. * The use of "os.CreateTemp" and "os.Remove" ensures that the test doesn't interfere with the actual filesystem and the temporary file is deleted after the test. * This example also uses t.Fatalf, which will terminate the test immediately allowing for quick debugging and error detection. * The example "loadModelConfig" is simplified for demonstration using hardcoded responses. In reality, this should parse and validate the YAML from the temporary file that was created. **Anti-Pattern:** """go // Anti-pattern: Unit test that performs I/O func TestLoadModelConfig_InvalidFile(t *testing.T) { // This test attempts to read a file directly, which is not a good practice // because it depends on the external environment. _, err := loadModelConfig("/path/to/nonexistent/file.yaml") // Dangerous I/O operation if err == nil { t.Errorf("Expected an error, but got nil") } } """ Instead, mock the loadModelConfig function to simulate a file not found error. ### 2.3. Mocking Libraries Use mocking libraries (like "testify/mock" in Go) to create mock implementations of interfaces and dependencies. This allows you to control the behavior of external components for testing purposes. """go // Example: Using testify/mock to mock an external service package ollama import ( "testing" "github.com/stretchr/testify/mock" ) // Define an interface that an external service implements type ModelLoader interface { LoadModel(modelName string) (string, error) } // Create a mock implementation of the ModelLoader interface type MockModelLoader struct { mock.Mock } // Implement the LoadModel method for the mock func (m *MockModelLoader) LoadModel(modelName string) (string, error) { args := m.Called(modelName) return args.String(0), args.Error(1) } // Function to test that uses the ModelLoader interface func UseModelLoader(loader ModelLoader, modelName string) (string, error) { return loader.LoadModel(modelName) } func TestUseModelLoader(t *testing.T) { // Create a new mock ModelLoader instance mockLoader := new(MockModelLoader) // Define what the mock should return when LoadModel is called with "test_model" mockLoader.On("LoadModel", "test_model").Return("model_content", nil) // Use the mock in our test result, err := UseModelLoader(mockLoader, "test_model") // Assert that the result and error are as expected if err != nil { t.Fatalf("Unexpected error: %v", err) } if result != "model_content" { t.Errorf("Expected 'model_content', but got '%s'", result) } // Assert that the mock's LoadModel method was called once with the correct argument mockLoader.AssertExpectations(t) } """ **Explanation:** * The MockModelLoader allows for controlling the exact outputs of the LoadModel, while asserting that the function was only called once. * This ensures only the function UseModelLoader is tested, and the outputs can be reliably checked. ## 3. Integration Testing Standards Integration tests verify the interaction between different components or modules of the Ollama system. These tests are more comprehensive than unit tests and help uncover issues related to component dependencies and communication. Specifically, focus on testing the integration of model loading with inference engines, API endpoints with authentication mechanisms, and various data storage components. ### 3.1. Guidelines * **Do This**: Test the integration of two or more components, focusing on data flow and interactions. * **Do This**: Use real dependencies where appropriate, but consider using test containers or lightweight substitutes to avoid external service dependencies. * **Do This**: Ensure that integration tests cover common use cases and error scenarios. * **Don't Do This**: Mock internal components excessively; the goal is to test real interactions. * **Don't Do This**: Write integration tests that are too broad and test the entire system; focus on specific integrations. * **Why**: Integration tests catch issues that unit tests might miss, such as incorrect data mapping or communication protocol errors. ### 3.2. Specific Implementations & Examples """go // Example: Integration test for loading a model and running a simple inference package ollama import ( "testing" // "ollama.ai/ollama/pkg/llm" // Use the actual llm package "os" "fmt" ) // Mock llm interface for demonstrating the integration test type LLM interface { Predict(prompt string) (string, error) } type MockLLM struct {} func (m *MockLLM) Predict(prompt string) (string, error) { return fmt.Sprintf("Response for prompt: %s", prompt), nil } //Simplified implementation that does not actually load a model func loadAndPredict(modelPath string, prompt string) (string, error) { llm := &MockLLM{} return llm.Predict(prompt) //Simlified to llm.Predict for the mock } func TestLoadModelAndPredict(t *testing.T) { // Create a model directory and a dummy model file modelDir := t.TempDir() modelFile := modelDir + "/test_model.bin" err := os.WriteFile(modelFile, []byte("dummy model content"), 0644) //Creating a dummy file instead of loading a real model if err != nil { t.Fatalf("Failed to create model file: %v", err) } // Define a test prompt prompt := "What is the capital of France?" // Call the integration function result, err := loadAndPredict(modelFile, prompt) // Assert that the result is as expected if err != nil { t.Fatalf("Unexpected error: %v", err) } expectedResult := "Response for prompt: What is the capital of France?" if result != expectedResult { t.Errorf("Expected '%s', but got '%s'", expectedResult, result) } } """ **Explanation:** * The test creates a temporary directory and a dummy model file to simulate a real model. (In a real implemntation, this would load an actual model using the Ollama libraries.) * It then calls the "loadAndPredict" function, which simulates loading the model and running a prediction. The mock avoids external calls for the sake of testing. * The test asserts that the result is as expected. * This example checks the integration between the model loading and inference components. **Anti-Pattern:** """go // Anti-pattern: Integration test that is too broad func TestEntireOllamaSystem(t *testing.T) { // This test attempts to test the entire Ollama system, which is not a good practice // because it is too broad and difficult to maintain. It is brittle and will break often when the backend implementation changes. // Instead, focus on specific integrations. // ... test logic that covers many modules ... } """ ## 4. End-to-End (E2E) Testing Standards End-to-end tests simulate real user interactions with the Ollama system. These tests verify that the entire system works correctly, from the user interface to the backend services. These tests validate critical workflows, such as deploying a new model, querying it via the API, and monitoring its performance. ### 4.1. Guidelines * **Do This**: Focus on critical user journeys and high-level scenarios. * **Do This**: Use a dedicated test environment that closely mirrors the production environment. * **Do This**: Automate E2E tests to run regularly as part of the CI/CD pipeline. * **Don't Do This**: Write E2E tests that are too granular or cover minor details; focus on the big picture. * **Don't Do This**: Neglect performance and security testing in E2E scenarios. * **Why**: E2E tests provide the highest level of confidence that the system works as expected in real-world scenarios. ### 4.2. Specific Implementations & Examples """go // Example: End-to-end test for deploying and querying a model package ollama import ( "testing" "os/exec" "net/http" "io/ioutil" "strings" "time" ) // Function to check if the Ollama server is running func isOllamaRunning() bool { resp, err := http.Get("http://localhost:11434/") // Assuming default port if err != nil { return false } defer resp.Body.Close() return resp.StatusCode == http.StatusOK } // Function to run Ollama commands func runOllamaCommand(command string) (string, error) { cmd := exec.Command("ollama", strings.Split(command, " ")...) out, err := cmd.CombinedOutput() return string(out), err } func TestDeployAndQueryModel(t *testing.T) { // Ensure Ollama server is running if !isOllamaRunning() { t.Skip("Ollama server is not running; skipping E2E test") } // 1. Deploy a model modelName := "test_model" _, err := runOllamaCommand("create " + modelName + " FROM 'FROM scratch\nCMD echo \'Hello, world!\''") //Simplified Modelfile if err != nil { t.Fatalf("Failed to deploy model: %v", err) } defer runOllamaCommand("rm " + modelName) // Cleanup after the test // 2. Give the server time to load the model time.Sleep(5 * time.Second) // 3. Query the model resp, err := http.Post("http://localhost:11434/api/generate", "application/json", strings.NewReader("{"prompt": "Say hello"}")) if err != nil { t.Fatalf("Failed to query model: %v", err) } defer resp.Body.Close() body, err := ioutil.ReadAll(resp.Body) if err != nil { t.Fatalf("Failed to read response body: %v", err) } // 4. Assert that the response is as expected expectedResponse := "Hello, world!" if !strings.Contains(string(body), expectedResponse) { t.Errorf("Expected response to contain '%s', but got '%s'", expectedResponse, string(body)) } } """ **Explanation:** * The test first checks if the Ollama server is running. * It then deploys a test model using the "ollama create" command with a simplified Modelfile. * It gives the server some time to load the model. * It then queries the model via the API and asserts that the response contains the expected content. * The test uses basic command execution and HTTP requests to simulate real user interactions. * The "defer runOllamaCommand("rm " + modelName)" ensures that the test cleans up after itself by removing the deployed model. **Anti-Pattern:** """go // Anti-pattern: E2E test that relies on specific UI elements func TestUIInteraction(t *testing.T) { // This test attempts to interact with specific UI elements, which is not a good practice // because it is brittle and will break easily when the UI changes. // Instead, focus on critical user journeys and high-level scenarios. // ... test logic that depends on specific UI elements ... } """ ### 4.3. Infrastructure and Tools * Utilize tools like Docker Compose to manage test environments with multiple Ollama instances and dependencies. * Consider using dedicated E2E testing frameworks like Cypress or Playwright for browser-based testing if Ollama has a web UI component. ## 5. Performance Testing Standards Performance testing is crucial for ensuring that Ollama can handle the required load and provide acceptable response times. These tests should measure metrics such as requests per second, latency, and resource utilization. ### 5.1. Guidelines * **Do This**: Define clear performance goals and metrics based on expected usage patterns. * **Do This**: Use load testing tools (e.g., Apache JMeter, k6) to simulate realistic traffic. * **Do This**: Monitor resource utilization (CPU, memory, network) during performance tests. * **Do This**: Identify and address performance bottlenecks through profiling and optimization. * **Don't Do This**: Perform performance tests in isolation; consider the impact of other services and dependencies. * **Don't Do This**: Neglect long-running tests to identify memory leaks or other stability issues. * **Why**: Performance tests ensure that Ollama can meet the performance requirements of its users and prevent performance degradation over time. ### 5.2. Specific Implementations & Examples """go // Example: Performance test using Apache JMeter // 1. Create a JMeter test plan (e.g., ollama_performance_test.jmx) that simulates // multiple users querying the Ollama API concurrently. The test plan should define // a range of variables, like thread count, ramp-up, and number of loops. // 2. Configure the JMeter test plan to send HTTP requests to the Ollama API endpoint // (e.g., http://localhost:11434/api/generate) with different prompts. // 3. Run the JMeter test plan from the command line: // jmeter -n -t ollama_performance_test.jmx -l ollama_performance_results.jtl // 4. Analyze the results in the ollama_performance_results.jtl file to identify // performance bottlenecks and areas for optimization. Use JMeter's reporting tools. //Example Code (Go) to measure a model's inference time. package ollama import ( "testing" "time" ) func benchmarkModelInference(b *testing.B, prompt string) { llm := &MockLLM{} //Using the mock LLM created in previous section b.ResetTimer() for i := 0; i < b.N; i++ { _, _ = llm.Predict(prompt) } } func BenchmarkModelInferenceShortPrompt(b *testing.B) { benchmarkModelInference(b, "Short prompt") } func BenchmarkModelInferenceLongPrompt(b *testing.B) { longPrompt := strings.Repeat("This is a long prompt. ", 100) benchmarkModelInference(b,longPrompt) } """ **Explanation:** * Benchmark tests are created for short and long prompts. These tests use the mock llm predict function to benchmark the function. * When running the "go test -bench=." command, go will run these benchmarks and provide information on performance. ## 6. Security Testing Standards Security testing is essential to identifying and mitigating security vulnerabilities in the Ollama system. These tests should cover areas such as authentication, authorization, input validation, and data protection. ### 6.1. Guidelines * **Do This**: Perform regular security audits and penetration tests to identify potential vulnerabilities. * **Do This**: Implement robust authentication and authorization mechanisms to protect access to sensitive resources. * **Do This**: Validate all user inputs to prevent injection attacks and other security exploits. * **Do This**: Encrypt sensitive data both in transit and at rest. * **Don't Do This**: Store sensitive information (e.g., API keys, passwords) in plaintext. * **Don't Do This**: Neglect security considerations during the design and development phases. * **Why**: Security tests protect the Ollama system from unauthorized access, data breaches, and other security threats. ### 6.2. Specific Implementations & Examples * Use tools like OWASP ZAP or Burp Suite to perform dynamic application security testing (DAST). * Implement static analysis tools (e.g., SonarQube, GoSec) to identify security vulnerabilities in the code. * Perform manual code reviews to identify security flaws that automated tools might miss. **Specific Ollama examples will depend on the final architecture designed.** As such, design guidance can be provided: * Authentication Testing: Test authentication failures (e.g. brute forcing login, bypassing authentication mechanisms, invalid or expired tokens), Authorization testing (e.g. privilege escalation, access to resources for unauthorized users), and testing the retrieval of authorized resources only. * Implement testing for input validation: Perform fuzzing on API parameters with unexpected values to test if the server handles them correctly, and test for injection attacks. Examine the API for various injection types (SQL, command). * Rate Limiting: Ensure rate limiting middleware is properly configured and prevents abuse, with tests that call rapid, repeated API requests to look for throttling. ## 7. Testing in CI/CD Pipeline Automate all types of tests (unit, integration, E2E, performance, security) in the CI/CD pipeline to ensure continuous feedback and prevent regressions from reaching production. ### 7.1. Guidelines * **Do This**: Integrate tests into the CI/CD pipeline to run automatically on every code change. * **Do This**: Use a dedicated testing environment in the CI/CD pipeline. * **Do This**: Fail the build if any tests fail. * **Do This**: Provide clear and actionable feedback to developers when tests fail. * **Why**: Automated testing in the CI/CD pipeline ensures that code changes are thoroughly tested before being deployed to production. ### 7.2. Specific Implementations & Examples * Use CI/CD tools (e.g., Jenkins, GitLab CI, GitHub Actions) to define test pipelines. * Configure the CI/CD pipeline to run unit tests, integration tests, and E2E tests in parallel to reduce build time. * Integrate performance and security testing into the CI/CD pipeline to identify performance regressions and security vulnerabilities early in the development process.
# Tooling and Ecosystem Standards for Ollama This document outlines the coding standards for the Ollama project related to tooling and the broader ecosystem. Following these guidelines will ensure a consistent, maintainable, and performant codebase across all Ollama components and integrations. These standards are designed to be used by both human developers and AI coding assistants. ## 1. Development Environment Setup ### Standard 1.1: Consistent Development Environment **Do This:** * Use a consistent and reproducible development environment. Define this via Docker, Nix, or similar tools to guarantee that developers have the same dependencies and versions. * Employ a "devcontainer.json" file for VS Code or similar configuration files for other IDEs like IntelliJ or other editors. This facilitates automated environment setup. **Don't Do This:** * Rely on manually installed dependencies. This creates inconsistencies and "works on my machine" problems. **Why:** A consistent environment minimizes integration issues and reduces the debugging time associated with environment-specific bugs. **Example ("devcontainer.json"):** """json { "name": "Ollama Dev Environment", "build": { "dockerfile": "Dockerfile.dev" }, "extensions": [ "golang.go", "ms-azuretools.vscode-docker" ], "settings": { "go.toolsManagement.checkForUpdates": "local", "go.useLanguageServer": true, "go.gopath": "/go" }, "mounts": [ "source=${localWorkspaceFolder},target=/go/src/github.com/ollama/ollama,type=bind,consistency=cached" ], "runArgs": [ "--security-opt", "apparmor=unconfined" ] } """ **Dockerfile.dev:** """dockerfile FROM golang:1.21-alpine AS builder WORKDIR /app COPY go.mod go.sum ./ RUN go mod download COPY . . RUN go build -o ollama ./cmd/ollama FROM alpine:latest WORKDIR /app COPY --from=builder /app/ollama . # Copy necessary binaries and scripts used by ollama (example) COPY --from=builder /app/scripts ./scripts EXPOSE 11434 CMD ["./ollama", "serve"] """ **Anti-Pattern:** Omitting a "devcontainer.json" or similar setup, leading to developers spending excessive time configuring their environments. ### Standard 1.2: Dependency Management **Do This:** * Use Go modules (or the project’s established dependency management tool) to manage dependencies. * Vendoring dependencies when appropriate to ensure reproducibility and isolation from external changes. * Regularly update dependencies to receive security patches and bug fixes, but always test after updating! **Don't Do This:** * Rely on globally installed packages or outdated dependency management techniques. * Ignore dependency updates or skip security vulnerability checks. **Why:** Robust dependency management prevents version conflicts and ensures that the project remains stable and secure. **Example:** """bash go mod tidy go mod vendor """ **Anti-Pattern:** Not running "go mod tidy" or "go mod vendor" before committing code. ## 2. Code Analysis and Formatting ### Standard 2.1: Automated Code Formatting **Do This:** * Use "gofmt" or "goimports" for automatic code formatting. Integrate it into your IDE and CI/CD pipeline. * Configure your editor to format code on save. **Don't Do This:** * Manually format code or ignore formatting errors. * Disable code formatting tools. **Why:** Consistent code formatting enhances readability and reduces the cognitive load when reviewing code. **Example:** """bash go fmt ./... goimports -w ./... """ **Configuration Example (VS Code):** """json { "go.formatTool": "goimports", "editor.formatOnSave": true } """ **Anti-Pattern:** Inconsistent indentation and spacing, making the code difficult to read. ### Standard 2.2: Linting and Static Analysis **Do This:** * Use "golangci-lint" with a predefined configuration file (".golangci.yml"). * Run linters as part of your CI/CD pipeline. * Address linting issues promptly. **Don't Do This:** * Ignore linting warnings or errors. * Disable linters without a valid reason. **Why:** Linting helps identify potential bugs, code smells, and style inconsistencies. **Example (".golangci.yml"):** """yaml run: deadline: 5m tests: false linters: enable: - gofmt - goimports - govet - revive - staticcheck - unconvert - misspell - unparam - unused """ **Running Linter:** """bash golangci-lint run ./... """ **Anti-Pattern:** Committing code with linting errors, indicating a lack of attention to detail. ## 3. Testing and Quality Assurance ### Standard 3.1: Comprehensive Unit Tests **Do This:** * Write unit tests for all non-trivial functions and methods. * Aim for high code coverage (e.g., >80%). Use coverage reports to identify gaps in testing. * Use table-driven tests for parameterized testing. **Don't Do This:** * Skip unit tests for complex logic. * Have low code coverage. * Write trivial tests that don't properly validate functionality. **Why:** Unit tests verify the correctness of individual components and prevent regressions. **Example:** """go package mypackage import "testing" func TestAdd(t *testing.T) { testCases := []struct { a, b, expected int }{ {1, 2, 3}, {0, 0, 0}, {-1, 1, 0}, } for _, tc := range testCases { result := Add(tc.a, tc.b) if result != tc.expected { t.Errorf("Add(%d, %d) = %d, expected %d", tc.a, tc.b, result, tc.expected) } } } """ **Anti-Pattern:** Writing tests that only check for error conditions and not the correct functionality. ### Standard 3.2: Integration Tests **Do This:** * Write integration tests to verify the interaction between different components of the system and external services (e.g., Ollama APIs, databases, etc.). * Use mocking frameworks to isolate components during testing. * Include tests that cover common use cases and error scenarios. **Don't Do This:** * Skip integration tests and assume components will work together correctly. * Rely solely on unit tests. **Why:** Integration tests ensure that the system functions correctly as a whole. **Example (using a mocking framework, conceptually):** """go package integrationtest import ( "testing" "net/http" //Example import "github.com/golang/mock/gomock" "github.com/ollama/ollama/api" // Hypothetical Ollama API package ) func TestMyIntegration(t *testing.T) { ctrl := gomock.NewController(t) defer ctrl.Finish() // Mock the Ollama API client //mockOllamaClient := api.NewMockOllamaClient(ctrl) //Example mocking and expectation //mockOllamaClient.EXPECT().PullModel(gomock.Any(), "my-model").Return(nil) // Your integration test logic here, using the mockOllamaClient // ... test code utilizing the Mock } """ **Anti-Pattern:** Integration tests that are too brittle and fail based on minor configuration changes. Avoid hardcoding IPs or other environment-specific settings. ### Standard 3.3: Performance and Load Testing **Do This:** * Conduct performance and load testing to identify potential bottlenecks and performance issues. * Use tools like "k6" or "vegeta" to simulate realistic workloads. * Monitor system metrics (e.g., CPU usage, memory usage, latency) during testing. * Integrate performance tests into CI/CD to detect performance regressions early. **Don't Do This:** * Ignore performance testing until late in the development cycle. * Fail to monitor system performance during testing. **Why:** Performance testing helps ensure that the system can handle the expected load and meet performance requirements. **Example (k6 script):** """javascript import http from 'k6/http'; import { check } from 'k6'; export const options = { vus: 10, duration: '10s', }; export default function () { const res = http.get('http://localhost:11434/api/tags'); check(res, { 'status is 200': (r) => r.status === 200, }); } """ **Running k6:** """bash k6 run script.js """ **Anti-Pattern:** Failing to establish baseline performance metrics and track performance changes over time. ## 4. Documentation ### Standard 4.1: Godoc Comments **Do This:** * Write clear and concise Godoc comments for all exported types, functions, and methods. * Include examples in your Godoc comments to illustrate how to use the code. * Keep Godoc comments up-to-date as the code evolves. **Don't Do This:** * Omit Godoc comments for exported APIs. * Write vague or misleading comments. * Let Godoc comments become stale. **Why:** Godoc comments provide essential documentation for users of the Ollama API. **Example:** """go // Add adds two integers and returns the result. // // Example: // // result := Add(1, 2) // fmt.Println(result) // Output: 3 func Add(a, b int) int { return a + b } """ **Anti-Pattern:** Using overly verbose comments that repeat what the code already expresses. ### Standard 4.2: Architecture and Design Documents. **Do This:** * Maintain high-level architecture and design documents, ideally in a tool like C4 or similar, to clearly outline components in Ollama and their integration. * Update these documents whenever there are significant changes to the system's architecture. Store them near the code! **Don't Do This:** * Let architecture documents become outdated * Fail to document key design decisions **Why:** Architecture and Design Documents ensures all developers have a clear overview of the project as a whole. ## 5. Logging and Monitoring ### Standard 5.1: Structured Logging **Do This:** * Use structured logging (e.g., with "zap" or "logrus") to log events in a consistent and machine-readable format. * Include relevant context in your logs (e.g., request ID, user ID, component name). * Log at appropriate levels (e.g., debug, info, warn, error, fatal). * Use log correlation IDs to track requests across multiple services. **Don't Do This:** * Use plain text logging, which is difficult to parse and analyze. * Log sensitive information (e.g., passwords, API keys). * Over-log or under-log events. **Why:** Structured logging simplifies log analysis and debugging. **Example (using "zap"):** """go package main import ( "go.uber.org/zap" ) func main() { logger, _ := zap.NewProduction() defer logger.Sync() // flushes buffer, if any sugar := logger.Sugar() sugar.Infow("failed to fetch URL", "url", "http://example.com", "attempt", 3, "err", err, ) } """ **Anti-Pattern:** Logging error messages without providing sufficient context to diagnose the problem. ### Standard 5.2: Application Monitoring **Do This:** * Use a monitoring system (e.g., Prometheus, Grafana) to track key metrics (e.g., request latency, error rates, resource usage). * Set up alerts for critical events (e.g., high error rates, resource exhaustion). * Use distributed tracing (e.g., Jaeger, Zipkin) to trace requests across multiple services. **Don't Do This:** * Fail to monitor application performance. * Ignore alerts. **Why:** Monitoring helps detect and diagnose problems promptly. **Example (Prometheus integration):** """go package main import ( "net/http" "github.com/prometheus/client_golang/prometheus/promhttp" ) func main() { http.Handle("/metrics", promhttp.Handler()) http.ListenAndServe(":2112", nil) } """ **Anti-Pattern:** Alerting on metrics that trigger too frequently and desensitize the team, or conversely setting thresholds too high so real issues are missed. ## 6. Security ### Standard 6.1: Secure Coding Practices **Do This:** * Follow secure coding practices to prevent common vulnerabilities (e.g., SQL injection, XSS, CSRF). * Validate all input data. * Use parameterized queries to prevent SQL injection. * Encode output data to prevent XSS. * Implement CSRF protection. **Don't Do This:** * Trust user input without validation. * Use dynamic SQL queries directly from user input. **Why:** Security is paramount to the integrity and trustworthiness of Ollama. ### Standard 6.2: Vulnerability Scanning **Do This:** * Use vulnerability scanning tools (e.g., "govulncheck") to identify security vulnerabilities in dependencies. * Regularly update dependencies to patch known vulnerabilities. **Don't Do This:** * Ignore vulnerability scans. * Fail to update dependencies with known vulnerabilities. **Why:** Proactive vulnerability scanning helps prevent security breaches. **Example:** """bash go install golang.org/x/vuln/cmd/govulncheck@latest govulncheck ./... """ **Anti-Pattern:** Ignoring vulnerability reports and accumulating vulnerable dependencies. ## 7. Ollama Specific Ecosystem Considerations ### Standard 7.1: Utilize Official Ollama Libraries and Tools **Do This:** * Prefer official Ollama libraries and tools when available. * Adhere to the documented APIs and usage patterns. * Contribute back to the official libraries when possible. **Don't Do This:** * Reinvent the wheel with custom solutions when official libraries exist. * Use undocumented features or APIs that may change without notice. **Why:** Using official libraries ensures compatibility and leverages the expertise of the Ollama project. **Example:** When interacting with the Ollama API, use the official Go client library (if it exists). If performing file operations on Ollama's data directory, use the provided utility functions, rather than directly manipulating files. ### Standard 7.2: Modelfile Best Practices **Do This:** * Follow the Modelfile documentation when creating custom models. * Use descriptive comments within the Modelfile to explain the model's configuration. * Test Modelfiles thoroughly to ensure they produce the desired results. * Use variables within the Modelfile for parameters, to make it more readable. * Consider using stages within the Modelfile to reduce the size of the final model by removing intermediate files. **Don't Do This:** * Use magic numbers or unclear configuration settings in your Modelfiles. * Create excessively large or complex Modelfiles without proper documentation. * Store sensitive data (e.g., API keys) directly in Modelfiles. **Why:** Well-written Modelfiles contribute to model reproducibility and ease of use. **Example:** """dockerfile # Modelfile for a CodeLlama model fine-tuned for Python code generation. FROM codellama # Set the temperature for code generation. PARAMETER temperature 0.7 # Install necessary Python libraries. RUN pip install numpy pandas requests # Add custom code generation scripts. COPY scripts/ /app/scripts/ # Run the code generation script. CMD ["python", "/app/scripts/generate_code.py"] """ ### Standard 7.3: Community Engagement **Do This:** * Engage with the Ollama community through forums, chat channels, and issue trackers. * Share your experiences and contribute to the project's knowledge base. * Provide feedback on existing tools and libraries. **Don't Do This:** * Be afraid to ask questions or seek help from the community. * Ignore community feedback on your code or contributions. **Why:** Community engagement fosters collaboration and accelerates the development of the Ollama ecosystem. By adhering to these coding standards, we can ensure that the Ollama project remains maintainable, performant, and secure, while also fostering a thriving ecosystem of tools and libraries.
# Code Style and Conventions Standards for Ollama This document outlines the coding style and conventions to be followed when contributing to the Ollama project. Adhering to these guidelines ensures code consistency, readability, and maintainability across the project. These standards are intended to guide developers, act as a reference for code reviews, and offer context for AI coding assistants. ## 1. General Principles ### 1.1. Consistency * **Do This:** Maintain consistency within a file, module, and across the entire project. Adopt existing patterns and styles. * **Don't Do This:** Introduce new and inconsistent styles without prior discussion and agreement. * **Rationale:** Inconsistent code reduces readability and increases cognitive load. By adhering to a uniform styling, the project becomes more accessible and easier to maintain. ### 1.2. Readability * **Do This:** Write code that is easy to understand at a glance. Use meaningful names, comments, and spacing. * **Don't Do This:** Write complex or obfuscated code that requires significant effort to understand. * **Rationale:** Readable code allows other developers (and your future self) to quickly grasp the logic and intent, reducing maintenance time and bugs introduced during modifications. ### 1.3. Simplicity * **Do This:** Aim for the simplest solution that meets the requirements. Avoid over-engineering. * **Don't Do This:** Introduce unnecessary complexity or abstractions. * **Rationale:** Simpler code is easier to test, debug, and maintain. It also reduces the likelihood of introducing bugs. ### 1.4. Clarity * **Do This:** Ensure the purpose of each code block (function, class, or module) is clear and easily understandable. Use comments to explain complex logic or non-obvious decisions. * **Don't Do This:** Leave segments of code without clearly defined purpose or without adequate documentation, forcing other developers to reverse engineer the intent. * **Rationale:** Clarity is crucial for collaboration. When the purpose and function of the code are immediately obvious the barrier to contribution is dramatically lowered thereby increasing collaboration and reducing debugging effort. ## 2. Language-Specific Conventions (Go) Ollama is primarily written in Go. Therefore, Go-specific conventions are very important. ### 2.1. Formatting * **Do This:** Use "gofmt" to automatically format your code. Configure your editor to run "gofmt" on save. * **Don't Do This:** Manually format code or ignore "gofmt" suggestions. * **Rationale:** "gofmt" enforces a single, consistent style across the codebase, eliminating debates about formatting. """go // Example of properly formatted Go code package main import "fmt" func main() { message := "Hello, Ollama!" fmt.Println(message) } """ ### 2.2. Naming Conventions * **Do This:** * Use "CamelCase" for exported types and functions. * Use "camelCase" for unexported types and functions. * Use meaningful and descriptive names. * Use acronyms in all caps (e.g., "APIURL", "HTTPClient"). * Follow Go's naming conventions for variables (e.g., "i" for loop counters, "err" for errors). * **Don't Do This:** * Use abbreviations that are not widely understood. * Use names that are too short or too long. * Use inconsistent naming styles. * Prefix variable names with types (Hungarian notation). * **Rationale:** Consistent naming improves readability and helps understand the purpose and scope of variables, types, and functions. """go // Example of good naming conventions type APIClient struct { APIURL string } func (c *APIClient) FetchData(resourceID string) ([]byte, error) { // ... } // Example of function with good meaningful name func processUserInput(inputString string) (string, error) { // ... } """ ### 2.3. Comments and Documentation * **Do This:** * Write clear and concise comments to explain complex logic or non-obvious decisions. * Document all exported types, functions, and constants using Go's documentation comments. * Use sentence-case for comments, starting with a capital letter and ending with a period. * **Don't Do This:** * Write comments that state the obvious. * Leave code undocumented. * Use informal language or abbreviations in comments. * **Rationale:** Good documentation is essential for understanding and maintaining the codebase. "godoc" extracts these comments to generate API documentation. Clarity regarding purpose and usage of the function helps other developers to quickly start using it. """go // APIClient is a client for interacting with the Ollama API. type APIClient struct { APIURL string // APIURL is the URL of the Ollama API. } // FetchData retrieves data from the Ollama API for the given resource ID. func (c *APIClient) FetchData(resourceID string) ([]byte, error) { // ... } """ ### 2.4. Error Handling * **Do This:** * Always check for errors. * Return errors as the last return value. * Use "errors.Is" and "errors.As" from the "errors" package (or "xerrors" for earlier Go versions) to check for specific errors. * Provide context to errors using "fmt.Errorf" or the "errors.Wrap" function (from "pkg/errors" if needed). * Log errors with sufficient context. * **Don't Do This:** * Ignore errors. * Panic unless absolutely necessary (e.g., unrecoverable initialization errors). * Use naked returns without considering the error value. * **Rationale:** Proper error handling prevents unexpected program behavior and makes debugging easier. Including context in error messages helps pinpoint the location and cause of errors. """go func fetchData(url string) ([]byte, error) { resp, err := http.Get(url) if err != nil { return nil, fmt.Errorf("failed to fetch data from %s: %w", url, err) } defer resp.Body.Close() if resp.StatusCode != http.StatusOK { return nil, fmt.Errorf("unexpected status code %d", resp.StatusCode) } body, err := io.ReadAll(resp.Body) if err != nil { return nil, fmt.Errorf("failed to read response body: %w", err) } return body, nil } func main() { data, err := fetchData("https://example.com/api/data") if err != nil { log.Printf("Error: %v", err) return } fmt.Println(string(data)) } """ ### 2.5. Concurrency * **Do This:** * Use goroutines and channels for concurrent operations. * Use "sync" package primitives to manage shared state. * Be mindful of race conditions and deadlocks. * Use context to manage the lifecycle of goroutines. * **Don't Do This:** * Share memory by communicating. * Ignore potential data races. * Forget to handle panics in goroutines. * **Rationale:** Go's concurrency features are powerful but require careful use. Avoiding data races and managing goroutine lifecycles are crucial for building reliable concurrent applications. """go package main import ( "fmt" "sync" ) func worker(id int, jobs <-chan int, results chan<- int, wg *sync.WaitGroup) { defer wg.Done() for j := range jobs { fmt.Printf("worker %d started job %d\n", id, j) // Simulate some work // time.Sleep(time.Second) fmt.Printf("worker %d finished job %d\n", id, j) results <- j * 2 } } func main() { const numJobs = 5 jobs := make(chan int, numJobs) results := make(chan int, numJobs) var wg sync.WaitGroup // Start three workers, initially blocked because there are no jobs. for w := 1; w <= 3; w++ { wg.Add(1) go worker(w, jobs, results, &wg) } // Send 5 jobs over the jobs channel, then close that channel to // indicate that's all the work we have. for j := 1; j <= numJobs; j++ { jobs <- j } close(jobs) // Wait for the workers to finish, then close the results channel. wg.Wait() close(results) // Another possible way to ensure all workers have completed would be: // go func() { // wg.Wait() // close(results) // }() //Finally we collect all the results of the work. for a := range results { fmt.Println(a) } } """ ### 2.6. Dependencies * **Do This:** * Use Go modules to manage dependencies. Lock in your dependencies by committing the "go.mod" and "go.sum" files * Vendor dependencies when necessary for reproducibility and isolation but it is not best practice. * Keep dependencies up to date. * Avoid unnecessary dependencies. * **Don't Do This:** * Manually manage dependencies. * Use vulnerable or unmaintained dependencies. * **Rationale:** Using Go modules ensures reproducible builds and simplifies dependency management. Keeping dependencies up-to-date helps prevent security vulnerabilities. ### 2.7 Keep small functions * **Do This:** Create functions that do one thing well and keep them concise. Aim for functions that are no more than 30-50 lines to improve code clarity. * **Don't Do This:** Create lengthy functions with many responsibilities. * **Rationale** Smaller functions increase readability and testability which in turn enhances code maintainability. ## 3. Ollama-Specific Conventions ### 3.1. Modelfiles * **Do This:** * Follow the [Modelfile documentation](https://github.com/ollama/ollama/blob/main/README.md) for correct syntax and usage. * Use descriptive comments to explain the purpose of each instruction. * Use consistent formatting for instructions and arguments. * Organize instructions logically. * Ensure commands within the Modelfile are idempotent wherever possible. * **Don't Do This:** * Use deprecated or unsupported instructions. * Create Modelfiles that are difficult to understand or maintain. * Hardcode sensitive information in Modelfiles. * **Rationale:** Modelfiles define how models are created and configured in Ollama. Following these conventions ensures that models can be built consistently and reliably. """dockerfile # Modelfile for CodeLlama FROM codellama:7b # Set the temperature for text generation PARAMETER temperature 0.7 # Add a system message SYSTEM """You are a helpful code assistant.""" # Install any necessary dependencies RUN pip install --upgrade pip RUN pip install pandas numpy # Define instruction to guide the model INSTRUCTION """Write me a function that outputs the fibonacci sequence""" """ ### 3.2. Model Management * **Do This:** * Use the "ollama" CLI for managing models. * Organize models into logical namespaces. * Use descriptive tags for models. * **Don't Do This:** * Manually manipulate model files. * Use generic or ambiguous model names. * Neglect to version control your models or configurations. * **Rationale:** Proper model management is essential for reproducibility and collaboration. Consistent naming and tagging make it easier to find and use the right models. ### 3.3. API Usage * **Do This:** * Use the Ollama API to interact with models programmatically. * Handle API errors gracefully. * Use appropriate timeouts and retries. * Sanitize input data before sending it to the API. * **Don't Do This:** * Bypass the API and directly access model files. * Ignore API errors. * Send untrusted data to the API without proper validation. * **Rationale:** The Ollama API provides a consistent and secure way to interact with models. Following these guidelines ensures that your applications are robust and secure. ### 3.4. Logging * **Do This:** * Use structured logging with consistent log levels (e.g., "debug", "info", "warn", "error"). * Include relevant context in log messages (e.g., request ID, user ID). * Log errors with stack traces. * Use a logging library that supports configurable output formats (e.g., JSON). * **Don't Do This:** * Use "fmt.Println" for logging. * Log sensitive information. * Log excessively or insufficiently. * **Rationale:** Structured logging makes it easier to analyze logs and diagnose problems. Consistent log levels and context help prioritize and filter log messages. """go import ( "log" ) func main() { log.Println("This is standard log") log.Printf("This log contains a string %s and an integer %d", "test", 10) log.Fatal("This is a fatal error") log.Panic("This is a panic") } """ ### 3.5 Data Security * **Do This:** * Sanitize all input data to prevent injection attacks. * Store sensitive data securely (e.g., using encryption or key vaults). * Follow security best practices for your programming language and platform. * Regularly review and update security measures. * **Don't Do This:** * Hardcode sensitive information in code or configuration files. * Trust user input without validation. * Ignore security warnings or vulnerabilities. * **Rationale:** Data security is critical to protect user data and prevent unauthorized access. Following security best practices helps minimize the risk of security breaches. ## 4. Testing ### 4.1. Unit Tests * **Do This:** Write unit tests for all functions and methods. Aim for high code coverage. Use table-driven tests to cover various scenarios. Mock external dependencies to isolate units of code. Run unit tests frequently (e.g. via CI/CD pipeline). * **Don't Do This:** Skip unit tests, especially for critical functionality. Write brittle tests that break easily with minor code changes. * **Rationale:** Unit Tests enable early bug detection, allow faster development cycles, promote better code design and increase code reliability in the long run. """go package main import "testing" func add(a, b int) int { return a + b } func TestAdd(t *testing.T) { testCases := []struct { a, b, expected int }{ {1, 2, 3}, {0, 0, 0}, {-1, 1, 0}, {-1, -1, -2}, } for _, tc := range testCases { result := add(tc.a, tc.b) if result != tc.expected { t.Errorf("add(%d, %d) = %d; expected %d", tc.a, tc.b, result, tc.expected) } } } """ ### 4.2. Integration Tests * **Do This:** Write integration tests to ensure that different components of the system work together correctly. Test interactions with external APIs and databases. * **Don't Do This:** Neglect integration tests, leading to undetected integration issues. Rely solely on unit tests, which may not catch problems that arise from interactions between components. * **Rationale:** Integration tests increase confidence that different parts of the system work correctly by testing them to ensure cohesion. ### 4.3. End-to-End Tests * **Do This:** Write end-to-end (E2E) tests to validate the entire system from start to finish. Simulate user interactions to ensure the system behaves as expected. * **Don't Do This:** Skip E2E tests, risking major issues in user-facing features. Make E2E tests too complex or brittle. * **Rationale:** End-to-end tests emulate user behaviour to assert functionality matches business requirements for the user. ### 4.4. Test Data * **Do This:** Generate or carefully select test data to cover all relevant edge cases. Keep test data separate from code. Store test data in a sensible and maintainable format. * **Don't Do This:** Use insufficient test data or ignore edge cases. Mix test data with code. * **Rationale:** Complete test data reduces errors and increases confidence in software quality. ## 5. Performance Optimization ### 5.1. Algorithmic Efficiency * **Do This:**: Choose appropriate algorithms and data structures for the task at hand. Analyze existing code for performance bottlenecks. Opt for algorithms with better time complexity when performance is critical. * **Don't Do This:** Use inefficient or brute-force algorithms without considering their performance implications. Neglect to profile code to identify performance bottlenecks. * **Rationale:** Efficient algorithms ensure quick execution and optimized resource use which is especially important in high-demand environments. ### 5.2. Memory Management * **Do This:** Minimize unnecessary memory allocation and deallocation. Reuse objects when appropriate. Use data structures, such as sync.Pool for temporary object storage. * **Don't Do This:** Create excessive temporary objects that increase garbage collection overhead. Leak memory by failing to release resources. * **Rationale:** Optimizing memory management reduces overheads, speeds up processes and makes systems more responsive. ### 5.3. Concurrency * **Do This:** Use goroutines to perform concurrent operations. Use sync.WaitGroup to manage the lifecycle of goroutines. Use channels for inter-goroutine communication * **Don't Do This:** Create race conditions or deadlocks by incorrectly managing concurrent processes. Overuse concurrency thus negating it's positive potential. * **Rationale:** Optimal concurrency leads to superior resource utilization, increased throughput and responsiveness in systems. ### 5.4. Code Profiling * **Do This:** Use profiling tools (e.g., go tool pprof) to identify performance bottlenecks. Profile code in realistic scenarios to locate areas for improvement. * **Don't Do This:** Make performance optimizations without profiling the code first. Ignore the results of profiling, leading to uninformed adjustments. * **Rationale:** Profiling helps developers precisely identify performance bottlenecks, enabling targeted and effective improvements. ## 6. Security Considerations ### 6.1. Input Validation * **Do This:** Validate and sanitize all user inputs to prevent injection attacks. Use prepared statements for database queries. Use a well-vetted library for input validation and sanitization. * **Don't Do This:** Trust user inputs without validation. Concatenate strings to build database queries. * **Rationale:** Thorough input validation prevents exploits that can compromise the system. ### 6.2. Authentication and Authorization * **Do This:** Establish robust authentication mechanisms to verify user identities. Implement authorization controls to restrict access based on user roles and permissions. Use established and secure authentication protocols (e.g., OAuth 2.0). * **Don't Do This:** Use weak or custom authentication schemes. Grant excessive permissions to users. Store critical data in ways which make it easy to view without authorisation. * **Rationale:** Ensuring strong authentication and authorization policies protects resources and prevent unauthorized access. ### 6.3. Secure Communication * **Do This:** Use HTTPS to protect data in transit. Support HTTPS for all web services and APIs. Enforce the use of TLS 1.3 or higher versions. * **Don't Do This:** use HTTP for sensitive communications. Permit outdated versions of TLS (e.g., TLS 1.0, TLS 1.1). * **Rationale:** Secure communication protocols prevents eavesdropping and tampering, safeguarding data while it is in transit. This document embodies the coding style and conventions that are necessary for contributing to the Ollama project.
# Core Architecture Standards for Ollama This document defines the coding standards for the core architecture of Ollama. It outlines the architectural patterns, project structure, and organization principles that all Ollama developers must adhere to. Following these standards will ensure maintainability, performance, and security of the Ollama project. These standards are based on the latest versions of Ollama features and best practices. ## 1. Fundamental Architectural Patterns ### 1.1 Microkernel Architecture **Standard:** Ollama's core should adhere to a microkernel architecture. The core should be kept small and focused, delegating functionalities to plugins or extension modules. * **Do This:** Keep the core Ollama repository lean, focusing on essential model loading, processing, and API serving functionalities. Implement most features as separate modules or plugins loaded dynamically. * **Don't Do This:** Bloat the core with features that can be implemented independently and loaded dynamically. **Why:** * **Maintainability:** Easier to maintain a small core. * **Extensibility:** Allows new features to be added without modifying the core. * **Testability:** Simpler core leads to easier testing. **Code Example (Go - Plugin Loading):** """go // core/main.go package main import ( "fmt" "plugin" ) func main() { // Load the plugin plug, err := plugin.Open("./plugins/myplugin.so") if err != nil { panic(err) } // Look up a symbol in the plugin sym, err := plug.Lookup("MyFunction") if err != nil { panic(err) } // Assert that the symbol is of the expected type myFunction, ok := sym.(func() string) if !ok { panic("unexpected type from plugin") } // Call the function result := myFunction() fmt.Println(result) } """ """go // plugins/myplugin/myplugin.go package main import "C" //export MyFunction func MyFunction() string { return "Hello from plugin!" } func main() {} // Required for plugin to compile """ **Anti-Pattern:** Directly integrating unrelated features into the core Ollama package, increasing complexity and coupling. ### 1.2 Modular Design **Standard:** Ollama should be structured into modular components with well-defined interfaces. * **Do This:** Break down functionalities into isolated modules with clear APIs. Use idiomatic Go packages to organize code within a module. * **Don't Do This:** Create monolithic modules or packages with tightly coupled functionalities. **Why:** * **Decoupling:** Reduces dependencies, making modifications easier. * **Reusability:** Modules can be reused in different parts of the system. * **Testability:** Easier to test individual modules in isolation. **Code Example (Go - Modular API handling):** """go // internal/api/server.go package api import ( "fmt" "net/http" "log" ) // Route defines a route within the API. type Route struct { Name string Method string Pattern string HandlerFunc http.HandlerFunc } // Routes defines the list of routes of our API type Routes []Route // NewRouter creates a new router. func NewRouter() *http.ServeMux { router := http.NewServeMux() // Example route definition var routes = Routes{ Route{ "Index", "GET", "/", func(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, "Hello, World!") }, }, Route{ "ModelsList", "GET", "/api/models", listModels, // assuming listModels is defined elsewhere in the package for modularity. }, } for _, route := range routes { router.HandleFunc(route.Pattern, route.HandlerFunc) } return router } // StartServer starts the API server. func StartServer(port string) { router := NewRouter() log.Printf("Starting server on port %s", port) log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", port), router)) } """ """go // internal/api/models.go package api import ( "encoding/json" "net/http" ) // Model represents a model available in Ollama type Model struct { Name string "json:"name"" Size int64 "json:"size"" } func listModels(w http.ResponseWriter, r *http.Request) { // In a real implementation, these models might come from a database or configuration models := []Model{ {Name: "llama2", Size: 2147483648}, // 2GB {Name: "mistral", Size: 1073741824}, // 1GB } WriteJSON(w,http.StatusOK,models) } // WriteJSON writes the interface i to the response body in JSON format. func WriteJSON(w http.ResponseWriter, status int, i interface{}, headers ...http.Header) error { w.Header().Set("Content-Type", "application/json") for _, h := range headers { for key, value := range h { for _, v := range value { w.Header().Add(key, v) } } } w.WriteHeader(status) enc := json.NewEncoder(w) enc.SetIndent("", " ") if err := enc.Encode(i); err != nil { return err } return nil } """ **Anti-Pattern:** Adding new functionalities directly to existing large files without considering creating new modules. ### 1.3 Event-Driven Architecture (Where applicable) **Standard:** Use event-driven architecture for asynchronous tasks and decoupling related components. * **Do This:** Implement an internal event bus for publishing and subscribing to events, allowing components to react to changes without direct dependencies. Use channels in Go for implementation. * **Don't Do This:** Tightly couple components using direct function calls for handling asynchronous operations. **Why:** * **Decoupling:** Components can operate independently without direct knowledge of each other. * **Scalability:** Easier to scale individual components that react to events. * **Responsiveness:** Improves system responsiveness by handling asynchronous operations. **Code Example (Go - Event Bus):** """go // internal/events/bus.go package events import ( "fmt" ) // Event is an interface for all events. type Event interface { Type() string Data() interface{} } // Bus is the event bus. type Bus struct { subscribers map[string][]chan Event } // NewBus creates a new event bus. func NewBus() *Bus { return &Bus{ subscribers: make(map[string][]chan Event), } } // Subscribe subscribes to an event type. func (b *Bus) Subscribe(eventType string, ch chan Event) { b.subscribers[eventType] = append(b.subscribers[eventType], ch) } // Publish publishes an event to all subscribers of that event type. func (b *Bus) Publish(event Event) { eventType := event.Type() if subs, ok := b.subscribers[eventType]; ok { for _, ch := range subs { // Non-blocking send to avoid deadlocks select { case ch <- event: fmt.Printf("Event %s published\n", eventType) default: fmt.Printf("Event %s dropped due to full channel\n", eventType) } } } } """ """go // internal/events/model_loaded_event.go package events // ModelLoadedEvent is an event that is published when a model is loaded. type ModelLoadedEvent struct { ModelName string ModelSize int64 } func (e ModelLoadedEvent) Type() string { return "model.loaded" } func (e ModelLoadedEvent) Data() interface{} { return e } """ """go // core/model_manager.go - Example usage (simplified): package main import ( "fmt" "time" "ollama/internal/events" // Adjust import path as needed ) func main() { bus := events.NewBus() // Subscriber modelLoadedChan := make(chan events.Event, 10) // Buffered channel bus.Subscribe("model.loaded", modelLoadedChan) go func() { for event := range modelLoadedChan { modelEvent, ok := event.Data().(events.ModelLoadedEvent) if !ok { fmt.Println("Invalid Model Loaded Event data") continue } fmt.Printf("Model loaded: %s, size: %d\n", modelEvent.ModelName, modelEvent.ModelSize) } }() // Simulate loading a model, then publishing an event go func() { time.Sleep(1 * time.Second) // Simulate loading time event := events.ModelLoadedEvent{ModelName: "llama2", ModelSize: 2147483648} bus.Publish(event) // Close channel after publishing to signal completion. close(modelLoadedChan) fmt.Println("Event published, Load Complete") }() time.Sleep(2 * time.Second) // Wait enough time to receive the Event } """ **Anti-Pattern:** Implementing direct calls from one module to another for asynchronous tasks, creating tight coupling. ## 2. Project Structure and Organization ### 2.1 Directory Structure **Standard:** Follow a consistent directory structure for the Ollama project. * **Do This:** Structure the project using a layout similar to below. * "cmd/ollama": Main application entry point. * "internal/": Private application code that shouldn't be imported by external packages. * "internal/api": API server implementation. * "internal/model": Model management logic (loading, unloading, etc.). * "internal/gpu": GPU related code * "internal/llm": Core LLM runtime code * "internal/events" : Event Bus implementation * "pkg/": Public packages that can be imported by external applications. Use sparingly for truly reusable components, favor "internal/". * "plugins/": Dynamically loaded plugins. * "tests/": Integration and end-to-end tests. * "scripts/" : Helper or automation scripts. * "hack/": Scripts and utilities for development, deployment, and maintenance. * "docs/": Documentation. * **Don't Do This:** Randomly place files without a clear organizational structure. Mix internal and external code. **Why:** * **Clarity:** Easier to navigate and understand the codebase. * **Maintainability:** Simplifies code maintenance and refactoring. * **Testability:** Promotes clear separation of concerns for testing purposes. ### 2.2 Package Naming **Standard:** Use clear and descriptive package names. * **Do This:** Name packages based on their functionality and purpose. Use short, meaningful names. * **Don't Do This:** Use generic names like "utils" or "helpers" without a clear purpose. Avoid vague or ambiguous names. **Why:** * **Readability:** Easier to understand the purpose of a package from its name. * **Maintainability:** Simplifies code maintenance and reduces naming conflicts. **Good Examples:** "api", "model", "gpu", "llm", "events". **Bad Examples:** "utils", "helpers", "common", "misc". ### 2.3 Dependency Management **Standard:** Manage dependencies using Go modules. * **Do This:** Use "go.mod" to declare dependencies. Update dependencies regularly using "go get -u all". Audit dependencies for security vulnerabilities. * **Don't Do This:** Manually manage dependencies or check in dependencies into the repository. **Why:** * **Reproducibility:** Ensures consistent builds across different environments. * **Security:** Helps track and manage security vulnerabilities in dependencies. * **Maintainability:** Simplifies dependency updates and management. * **Correct versioning:** Ensures that the correct versions of dependencies are used, preventing unexpected behavior. **Code Example (Go - go.mod):** """mod module ollama go 1.21 require ( github.com/go-chi/chi/v5 v5.0.11 github.com/google/uuid v1.6.0 github.com/joho/godotenv v1.5.1 github.com/spf13/cobra v1.8.0 ) require ( github.com/davecgh/go-spew v1.1.1 // indirect github.com/fsnotify/fsnotify v1.7.0 // indirect github.com/inconshreveable/mousetrap v1.1.0 // indirect github.com/pmezard/go-difflib v1.0.0 // indirect github.com/sagikazarmark/locafero v0.4.0 // indirect github.com/sagikazarmark/slog-shim v0.1.0 // indirect github.com/sourcegraph/conc v0.3.0 // indirect github.com/spf13/afero v1.11.0 // indirect github.com/spf13/cast v1.6.0 // indirect github.com/spf13/pflag v1.0.5 // indirect github.com/stretchr/objx v0.5.0 // indirect github.com/stretchr/testify v1.9.0 // indirect go.uber.org/atomic v1.9.0 // indirect go.uber.org/multierr v1.9.0 // indirect golang.org/x/exp v0.0.0-20230905200255-9212866ba22d // indirect golang.org/x/net v0.20.0 // indirect golang.org/x/sys v0.16.0 // indirect golang.org/x/text v0.14.0 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect ) """ ## 3. Core Implementation Details ### 3.1 Model Loading and Management **Standard:** Implement robust and efficient model loading and management. * **Do This:** Use a dedicated "model" package for model loading and unloading. Implement caching mechanisms to avoid redundant loading. Support various model formats and architectures. Use the "internal/gpu" package for GPU management. * **Don't Do This:** Load models directly within API handlers. Hardcode model paths. **Why:** * **Performance:** Efficient model loading is crucial for responsiveness. Reuse loaded models. * **Flexibility:** Supports different model formats and hardware configurations. * **Maintainability:** Centralized model management simplifies updates and changes. **Code Example (Go - Model Loading):** """go // internal/model/manager.go package model import ( "fmt" "sync" ) // ModelManager manages loaded models. type ModelManager struct { models map[string]interface{} // Replace interface{} with the actual model type mu sync.RWMutex } // NewModelManager creates a new ModelManager. func NewModelManager() *ModelManager { return &ModelManager{ models: make(map[string]interface{}), } } // LoadModel loads a model. func (m *ModelManager) LoadModel(name string, path string) error { m.mu.Lock() defer m.mu.Unlock() // Check if the model is already loaded if _, ok := m.models[name]; ok { return fmt.Errorf("model %s already loaded", name) } // Simulate loading the model from disk // In a real implementation, this would involve reading the model from a file // and initializing it. fmt.Printf("Loading model %s from %s...\n", name, path) // Replace with your actual model loading logic modelData := fmt.Sprintf("Model data for %s", name) m.models[name] = modelData fmt.Printf("Model %s loaded successfully\n", name) return nil } // GetModel returns a loaded model. func (m *ModelManager) GetModel(name string) (interface{}, error) { m.mu.RLock() defer m.mu.RUnlock() model, ok := m.models[name] if !ok { return nil, fmt.Errorf("model %s not found", name) } return model, nil } // UnloadModel unloads a model. func (m *ModelManager) UnloadModel(name string) error { m.mu.Lock() defer m.mu.Unlock() if _, ok := m.models[name]; !ok { return fmt.Errorf("model %s not found", name) } delete(m.models, name) fmt.Printf("Model %s unloaded successfully\n", name) return nil } """ """go // internal/api/server.go package api import ( "net/http" "ollama/internal/model" // Correct import path. ) // ModelHandler handles requests related to models(simplified to show loading): func ModelHandler(modelManager *model.ModelManager) http.HandlerFunc { return func(w http.ResponseWriter, r *http.Request) { if r.Method == http.MethodPost { // Load model route /api/models + POST modelName := r.URL.Query().Get("name") modelPath := r.URL.Query().Get("path") err := modelManager.LoadModel(modelName, modelPath) if err != nil { http.Error(w, err.Error(), http.StatusInternalServerError) return } w.WriteHeader(http.StatusOK) w.Write([]byte("Model loaded successfully")) } else { http.Error(w, "Method not allowed", http.StatusMethodNotAllowed) } } } """ **Anti-Pattern:** Directly loading model from API handlers. ### 3.2 API Design **Standard:** Design RESTful APIs following best practices. * **Do This:** Use standard HTTP methods (GET, POST, PUT, DELETE). Return appropriate HTTP status codes. Use JSON for request and response bodies. Implement proper error handling and validation. Use a consistent API versioning scheme. * **Don't Do This:** Create inconsistent API endpoints or use non-standard HTTP methods. **Why:** * **Interoperability:** Enables easy integration with other systems. * **Maintainability:** Simplifies API updates and versioning. * **Usability:** Provides a clear and consistent API for developers. **Code Example (Go - API handler):** """go // internal/api/server.go package api import ( "encoding/json" "net/http" ) // Define a struct to represent the request payload type RequestPayload struct { Prompt string "json:"prompt"" } // Define a struct to represent the response payload type ResponsePayload struct { Response string "json:"response"" } // Generate handles the /api/generate endpoint. func Generate(w http.ResponseWriter, r *http.Request) { if r.Method != http.MethodPost { http.Error(w, "Method not allowed", http.StatusMethodNotAllowed) return } // Decode the request payload var payload RequestPayload err := json.NewDecoder(r.Body).Decode(&payload) if err != nil { http.Error(w, err.Error(), http.StatusBadRequest) return } // Process the prompt (replace with your actual LLM processing logic) response := processPrompt(payload.Prompt) // Create the response payload responsePayload := ResponsePayload{ Response: response, } // Encode the response payload as JSON w.Header().Set("Content-Type", "application/json") err = json.NewEncoder(w).Encode(responsePayload) if err != nil { http.Error(w, err.Error(), http.StatusInternalServerError) return } } // processPrompt simulates processing a prompt with an LLM. func processPrompt(prompt string) string { // Replace with your actual LLM processing logic return "Generated response for prompt: " + prompt } """ **Anti-Pattern:** Returning HTTP 200 OK for all requests, even when errors occur. ### 3.3 Error Handling **Standard:** Implement comprehensive error handling throughout the system. * **Do This:** Use Go's error handling mechanisms. Log errors with sufficient context for debugging. Return meaningful error messages to the client. Implement error recovery strategies where appropriate. Use custom error types where necessary. * **Don't Do This:** Ignore errors or silently fail. Return generic error messages without context. **Why:** * **Reliability:** Ensures the system handles errors gracefully. * **Debuggability:** Provides detailed information for troubleshooting. * **Usability:** Helps users understand and resolve issues. **Code Example (Go - Error Handling):** """go // internal/api/server.go package api import ( "fmt" "log" "net/http" ) // Custom error type for model loading type ModelLoadError struct { ModelName string Err error } func (e *ModelLoadError) Error() string { return fmt.Sprintf("failed to load model %s: %v", e.ModelName, e.Err) } // LoadModelFromPath attempts to load a model from the given path. func LoadModelFromPath(path string) error { // Simulate a model loading failure if path == "" { return &ModelLoadError{ModelName: "ExampleModel", Err: fmt.Errorf("model path is empty")} } return nil } // handler function func loadModelHandler(w http.ResponseWriter, r *http.Request) { modelPath := r.URL.Query().Get("path") err := LoadModelFromPath(modelPath) if err != nil { // Log the error with context. log.Printf("Error loading model from path %s: %v", modelPath, err) // More detailed error message based on the error type. switch err.(type) { case *ModelLoadError: http.Error(w, fmt.Sprintf("Failed to load model: %s", err), http.StatusInternalServerError) default: http.Error(w, "Internal Server Error", http.StatusInternalServerError) } return } fmt.Fprintln(w, "Model loaded successfully.") } """ **Anti-Pattern:** Using "panic" for non-fatal errors. ### 3.4 Logging **Standard:** Use structured logging to record events and errors. * **Do This:** Use a logging library such as "zap" or "logrus". Log events at different levels (debug, info, warn, error). Include relevant context in log messages. Configure logging output (console, file, etc.). * **Don't Do This:** Use "fmt.Println" for logging. Log sensitive information. **Why:** * **Debuggability:** Provides detailed information for troubleshooting. * **Auditability:** Enables tracking of events and activities. * **Monitoring:** Allows monitoring of system health and performance. **Code Example (Go - Logging with "zap"):** """go // internal/logging/logger.go package logging import ( "go.uber.org/zap" "go.uber.org/zap/zapcore" "log" "os" ) var Logger *zap.Logger func InitLogger() { atom := zap.NewAtomicLevel() // Set the desired log level. Use environment variables to control this in production. atom.SetLevel(zap.DebugLevel) encoderConfig := zap.NewProductionEncoderConfig() encoderConfig.EncodeTime = zapcore.ISO8601TimeEncoder fileEncoder := zapcore.NewJSONEncoder(encoderConfig) consoleEncoder := zapcore.NewConsoleEncoder(encoderConfig) //Log file logFile, err := os.OpenFile("ollama.log", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644) if err != nil { log.Fatalf("Failed to open log file: %v", err) } core := zapcore.NewTee( zapcore.NewCore(fileEncoder, zapcore.AddSync(logFile), atom), //file Logger zapcore.NewCore(consoleEncoder, zapcore.AddSync(os.Stdout), atom), //console Logger ) Logger = zap.New(core, zap.AddCaller(), zap.AddStacktrace(zap.ErrorLevel)) // Add caller information and stack trace for errors defer Logger.Sync() Logger.Info("Logger initialized successfully") } """ """go // Example Usage package main import ( "ollama/internal/logging" "errors" // Standard error package ) func main() { logging.InitLogger() defer logging.Logger.Sync() // flushes buffer, if any logging.Logger.Info("Starting the application...") // Simulate an error err := errors.New("simulated error occurred") // Log the error with additional context logging.Logger.Error("Failed to process request", zap.Error(err), // Pass the error itself zap.String("request_id", "12345"), // Include request context for correlation zap.String("component", "data_loader"), // Indicate which component had the issue ) logging.Logger.Debug("Detailed debug message", zap.Int("value", 42)) logging.Logger.Warn("Configuration value is deprecated", zap.String("config_key", "old_setting")) logging.Logger.Info("Application finished") } """ **Anti-Pattern:** Logging sensitive data, such as passwords or API keys. ### 3.5 Concurrency **Standard:** Use Go's concurrency features safely and effectively. * **Do This:** Use goroutines for concurrent tasks. Use channels for communication between goroutines. Use mutexes or atomic operations for shared state. Avoid race conditions and deadlocks. Use errgroup for managing groups of goroutines. Leverage worker pools to limit concurrency. * **Don't Do This:** Share memory without synchronization. Launch unbounded numbers of goroutines. **Why:** * **Performance:** Enables efficient utilization of resources. * **Responsiveness:** Improves system responsiveness by handling concurrent tasks. * **Scalability:** Allows scaling the system by adding more resources. **Code Example (Go - Worker pool with errgroup):** """go // internal/workers/worker_pool.go package workers import ( "context" "fmt" "sync" "golang.org/x/sync/errgroup" ) // Task represents a unit of work to be done by a worker. type Task func() error // WorkerPool manages a pool of worker goroutines that execute tasks. type WorkerPool struct { numWorkers int jobQueue chan Task errGroup *errgroup.Group ctx context.Context cancel context.CancelFunc wg sync.WaitGroup // WaitGroup to wait for all workers to finish } // NewWorkerPool creates a new WorkerPool with the specified number of workers. func NewWorkerPool(numWorkers int) *WorkerPool { ctx, cancel := context.WithCancel(context.Background()) // Root Context group, ctx := errgroup.WithContext(ctx) return &WorkerPool{ numWorkers: numWorkers, jobQueue: make(chan Task), // Unbuffered channel: the sender will block until the receiver is ready errGroup: group, // Using errgroup for error propogation and context cancellation ctx: ctx, cancel: cancel, } } // Start starts the worker pool, launching the worker goroutines. func (wp *WorkerPool) Start() { for i := 0; i < wp.numWorkers; i++ { wp.wg.Add(1) // Increment the WaitGroup counter for each worker wp.errGroup.Go(func() error { defer wp.wg.Done() // Decrement the counter when the worker exits for task := range wp.jobQueue { // Receive tasks from the jobQueue channel until it is closed select { case <-wp.ctx.Done(): // Check if the context has been cancelled. fmt.Println("Worker shutting down due to context cancellation") return wp.ctx.Err() // Exit the worker goroutine default: // Execute the Task if err := task(); err != nil { // Execute the task fmt.Printf("Task failed: %v\n", err) return err // Return the error from the task } } } fmt.Println("Worker exiting: no more jobs on jobQueue") return nil // Worker exiting normally }) } } // Submit submits a task to the worker pool. func (wp *WorkerPool) Submit(task Task) { select { case wp.jobQueue <- task: // Send the task to the jobQueue channel case <-wp.ctx.Done(): // If the context is cancelled, discard the task. fmt.Println("Task submission cancelled: context done") } } // Stop stops the worker pool, preventing new tasks from being submitted and waiting // for all workers to finish processing existing tasks before exiting. func (wp *WorkerPool) Stop() error { wp.cancel() // Cancel the context, signalling workers to stop close(wp.jobQueue) // Close the jobQueue. This will cause workers to exit their loop wp.wg.Wait() // Wait for all workers to complete their tasks before continuing. fmt.Println("All workers finished.") err := wp.errGroup.Wait() // Return the first non-nil error (if any) from any of the workers if err != nil && err != context.Canceled { fmt.Printf("Error group returned an error: %v\n", err) return err } return nil } """ """go //Example Usage: package main import ( "fmt" "time" "errors" // Import the errors package "ollama/internal/workers" // Adjust based on where worker_pool.go is located ) func main() { numWorkers := 3 wp := workers.NewWorkerPool(numWorkers) wp.Start() defer wp.Stop() // Submit some tasks for i := 1; i <= 5; i++ { taskID := i wp.Submit(func() error { fmt.Printf("Task %d: starting\n", taskID) time.Sleep(time.Duration(taskID) * time.Second) // Simulate variable processing time fmt.Printf("Task %d: completed\n", taskID) // Simulate an error on task 3 if taskID == 3 { return errors.New("simulated error in task 3") // Return a non-nil error } return nil // Return nil on successful completion }) } time.Sleep(10 * time.Second) wp.Stop() fmt.Println("Finished submitting tasks and waiting for workers") } """ **Anti-Pattern:** Using global variables without proper synchronization in concurrent code. ### 3.6 Security **Standard:** Implement security best practices throughout the system. * **Do This:** Validate user inputs. Sanitize data before using it. Use secure coding practices. Protect against common web vulnerabilities (e.g., XSS, CSRF, SQL injection). Implement authentication and authorization. Use encryption for sensitive data. Keep dependencies up-to-date to address vulnerabilities. * **Don't Do This:** Trust user inputs without validation. Store sensitive data in plain text. **Why:** * **Data Integrity:** Prevents data corruption and unauthorized access. * **Confidentiality:** Protects sensitive information from disclosure. * **Availability:** Ensures the system remains available and responsive. **Code Example (Go - Input validation):** """go // internal/api/server.go package api import ( "encoding/json" "fmt" "net/http" "strings" ) // Define a struct to represent the request payload type RequestPayload struct { Prompt string "json:"prompt"" } // ValidateRequest validates the request payload. func ValidateRequest(payload *RequestPayload) error { // Check if the prompt is empty. if payload.Prompt == "" { return fmt.Errorf("prompt cannot be empty") } // Check if the prompt is too long. if len(payload.Prompt) > 1000 { return fmt.Errorf("prompt is too long (maximum 1000 characters)") } // Sanitize the prompt to prevent XSS attacks. payload.Prompt = strings.ReplaceAll(payload.Prompt, "<", "<") payload.Prompt = strings.ReplaceAll(payload.Prompt, ">", ">") return nil } // Generate handles the /api/generate endpoint. func Generate(w http.ResponseWriter, r *http.Request) { if r.Method != http.MethodPost { http.Error(w, "Method not allowed", http.StatusMethodNotAllowed) return } // Decode the request payload var payload RequestPayload err := json.NewDecoder(r.Body).Decode(&payload) if err != nil { http.Error(w, err.Error(), http.StatusBadRequest) return } // Validate the request payload err = ValidateRequest(&payload) if err != nil { http.Error(w, err.Error(), http.StatusBadRequest) return } // Process the prompt (replace with your actual LLM processing logic) response := processPrompt(payload.Prompt) // Encode the response payload as JSON w.Header().Set("Content-Type", "application/json") json.
# Component Design Standards for Ollama This document outlines the component design standards for the Ollama project. It serves as a guide for developers to create reusable, maintainable, and performant components within the Ollama ecosystem. These standards promote consistency and improve the overall quality of the codebase. These standards should be used when using AI code assistants like GitHub Copilot or Cursor. ## 1. Architectural Principles ### 1.1 Modularity and Loose Coupling **Standard:** Components should be designed as independent modules with well-defined interfaces. Dependencies between components should be minimized. **Why:** Modularity allows for easier testing, modification, and reuse of components. Loose coupling reduces the impact of changes in one component on other parts of the system. **Do This:** * Define clear and concise interfaces for each component. * Use dependency injection to manage dependencies between components. * Favor composition over inheritance to achieve code reuse. **Don't Do This:** * Create tightly coupled components that are difficult to isolate and test. * Introduce circular dependencies between components. * Hardcode dependencies within components. **Example (Go):** """go // Correct: Using interfaces for loose coupling package model // ModelInterface defines the methods a model component should implement. type ModelInterface interface { LoadModel(path string) error Predict(input string) (string, error) UnloadModel() error } // OllamaModel implements ModelInterface type OllamaModel struct { modelPath string // Model specific implementation details ... } func (m *OllamaModel) LoadModel(path string) error { // Implements loading the model from the specified path m.modelPath = path // Implementation... return nil } func (m *OllamaModel) Predict(input string) (string, error) { // Implements processing and getting the model prediction // Implementation... return "predicted output", nil } func (m *OllamaModel) UnloadModel() error { // Implements model unloading and memory freeing // Implementation... return nil } // Incorrect: Tight coupling (avoid this) package model type OllamaModel struct { // Directly dependent on the "logger" package. This makes testing difficult. logger *logger.Logger // Assuming logger is another internal package modelPath string } func (m *OllamaModel) LoadModel(path string, logger *logger.Logger) error { m.modelPath = path m.logger = logger m.logger.Log("Loading model from path: " + path) // Implementation... return nil } """ ### 1.2 Separation of Concerns **Standard:** Each component should have a single, well-defined responsibility. **Why:** Separation of concerns improves code readability, maintainability, and testability. It allows developers to focus on specific aspects of the system. **Do This:** * Divide the system into components based on their functional roles (e.g., data access, business logic, presentation). * Ensure that each component performs only the tasks related to its responsibility. * Minimize the overlap in functionality between components. **Don't Do This:** * Create "god classes" or components that perform too many unrelated tasks. * Mix different concerns within the same component (e.g., mixing data validation with business logic). **Example (Go):** """go // Correct: Separate components for handling requests and model inference package service import ( "fmt" "net/http" "ollama.ai/model" // Assuming ollama.ai/model package ) // InferenceHandler handles HTTP requests for model inference. type InferenceHandler struct { Model model.ModelInterface // Use the interface here for loose coupling } // NewInferenceHandler creates new InferenceHandler instance func NewInferenceHandler(model model.ModelInterface) *InferenceHandler { return &InferenceHandler{Model: model} } func (h *InferenceHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) { // Request parsing, validation, and pre-processing input := r.URL.Query().Get("input") if input == "" { http.Error(w, "Input parameter is required", http.StatusBadRequest) return } // Call to the Model component for inference output, err := h.Model.Predict(input) if err != nil { http.Error(w, fmt.Sprintf("Model prediction error: %v", err), http.StatusInternalServerError) return } // Response formatting and sending w.WriteHeader(http.StatusOK) w.Write([]byte(output)) } // Incorrect: Mixing Concerns package service import ( "fmt" "net/http" "ollama.ai/model" ) // InferenceHandler (Bad Design) handles requests AND model inference. type InferenceHandler struct { Model model.OllamaModel // Directly using the concrete type making testing harder } func (h *InferenceHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) { // Request parsing, validation, pre-processing, and model inference ALL IN ONE PLACE. input := r.URL.Query().Get("input") if input == "" { http.Error(w, "Input parameter is required", http.StatusBadRequest) return } output, err := h.Model.Predict(input) // Directly calling Model's implementation if err != nil { http.Error(w, fmt.Sprintf("Model prediction error: %v", err), http.StatusInternalServerError) return } w.WriteHeader(http.StatusOK) w.Write([]byte(output)) } """ ### 1.3 Single Source of Truth (SSOT) **Standard:** Each piece of data or logic should have a single, authoritative source within the system. **Why:** SSOT prevents inconsistencies and reduces the risk of errors when data or logic needs to be updated. **Do This:** * Identify the authoritative source for each piece of data. * Ensure that all components access data from the single source of truth. * Avoid duplicating data or logic across multiple components. **Don't Do This:** * Store the same data in multiple places without proper synchronization. * Implement the same logic in multiple components. * Allow components to modify data without going through the authoritative source. **Example (Conceptual):** In Ollama, model definitions should have a single, authoritative source, which could be a dedicated configuration file or database. Components that need information about a model (e.g., its architecture, dependencies) should query this source rather than maintaining their own copies of the data. ### 1.4 Abstraction **Standard:** Hide complex implementation details behind simple, well-defined interfaces. **Why:** Abstraction simplifies the usage of components and reduces the impact of internal changes. **Do This:** * Create abstract interfaces that hide implementation details. * Use factory patterns to create instances of components without exposing concrete classes. * Provide clear and concise documentation for the public interfaces of components. **Don't Do This:** * Expose internal implementation details through public interfaces. * Create leaky abstractions that require users to be aware of the underlying implementation. * Over-abstract components, making them overly complex and difficult to use. ## 2. Component Implementation ### 2.1 Naming Conventions **Standard:** Use consistent and descriptive naming conventions for components, classes, methods, and variables. **Why:** Consistent naming improves code readability and maintainability. **Do This:** * Use CamelCase for class names (e.g., "OllamaModel", "RequestHandler"). * Use camelCase for method and variable names (e.g., "loadModel()", "modelPath"). * Use descriptive names that clearly indicate the purpose of the component, class, method, or variable. * Follow the Go naming conventions (e.g., use short, concise names; avoid abbreviations unless they are widely understood). **Don't Do This:** * Use cryptic or ambiguous names. * Use inconsistent naming conventions. * Use names that are too long or verbose. ### 2.2 Error Handling **Standard:** Handle errors gracefully and provide informative error messages. **Why:** Proper error handling prevents crashes and makes it easier to diagnose and resolve problems. **Do This:** * Return errors from functions and methods that can fail. * Check for errors and handle them appropriately (e.g., log the error, return a default value, or propagate the error to the caller). * Provide informative error messages that include context about the error. * Use the "errors" package in Go for creating and handling errors. * Consider using custom error types to provide more specific information about the error. **Don't Do This:** * Ignore errors. * Panic or exit the program without handling the error. * Provide vague or uninformative error messages. **Example (Go):** """go package model import ( "fmt" "errors" ) // Custom error type for model loading failures var ErrModelLoadingFailed = errors.New("failed to load model") type OllamaModel struct { modelPath string } func (m *OllamaModel) LoadModel(path string) error { if path == "" { return fmt.Errorf("model path cannot be empty: %w", ErrModelLoadingFailed) // Wrapping standard error } // Attempt to load the model. err := m.loadFromFile(path) if err != nil { return fmt.Errorf("error loading model from file: %w", err) // Wrapping the underlying error } m.modelPath = path return nil } func (m *OllamaModel) loadFromFile(path string) error { //Simulate loading the model and failing return errors.New("could not read model file") } //Usage example func main() { model := OllamaModel{} err := model.LoadModel("") if err != nil { fmt.Println(err) // prints: model path cannot be empty: failed to load model fmt.Println(errors.Is(err, ErrModelLoadingFailed)) // prints true - showing error wrapping } } """ ### 2.3 Logging **Standard:** Log important events and errors to aid in debugging and monitoring. **Why:** Logging provides valuable insights into the behavior of the system and helps to identify and resolve issues. **Do This:** * Use a logging library (e.g., "logrus", "zap") to log messages. * Log messages at the appropriate level (e.g., debug, info, warn, error). * Include context in log messages (e.g., component name, request ID). * Log errors and warnings along with stack traces. * Consider using structured logging to make it easier to analyze log data. **Don't Do This:** * Log too much or too little information. * Log sensitive information (e.g., passwords, API keys). * Use "fmt.Println" for logging (use a dedicated logging library instead). **Example (Go):** """go package main import ( "time" "github.com/sirupsen/logrus" ) var log = logrus.New() func main() { // You could set this to any io.Writer, see below. log.Out = os.Stdout // You can change the level of logging log.SetLevel(logrus.InfoLevel) log.WithFields(logrus.Fields{ "animal": "walrus", "size": 10, }).Info("A group of walrus emerges from the ocean") log.WithFields(logrus.Fields{ "omg": true, "number": 122, }).Warn("The group's number increased tremendously!") log.WithFields(logrus.Fields{ "omg": true, "number": 100, "id": "ollama-001", //example }).Error("Failed to process request") // Log an error with context time.Sleep(2 * time.Second) } """ ### 2.4 Concurrency **Standard:** Handle concurrency safely and efficiently. **Why:** Ollama often deals with concurrent requests and operations. Proper concurrency management is crucial for performance and stability. **Do This:** * Use goroutines and channels for concurrent execution. * Use mutexes or other synchronization primitives to protect shared resources. * Avoid race conditions and deadlocks. * Use the "sync" package in Go for managing concurrency. * Consider using worker pools to limit the number of concurrent goroutines. **Don't Do This:** * Share mutable state between goroutines without proper synchronization. * Create too many goroutines, which can lead to performance problems. * Block indefinitely in goroutines, which can lead to deadlocks. ### 2.5 Resource Management **Standard:** Manage resources (e.g., memory, file handles, network connections) carefully to prevent leaks. **Why:** Resource leaks can lead to performance degradation and instability. **Do This:** * Use "defer" to ensure that resources are released when they are no longer needed. * Close file handles and network connections explicitly. * Free memory that is no longer being used. * Use garbage collection efficiently (e.g., avoid creating unnecessary objects). **Don't Do This:** * Forget to release resources. * Allocate large amounts of memory without a clear need. * Create circular references that prevent garbage collection. **Example (Go):** """go package main import ( "fmt" "os" ) func main() { // Open a file. file, err := os.Open("my_file.txt") if err != nil { fmt.Println("Error opening file:", err) return } defer file.Close() //Ensure file is closed when function exits. // Read from the file. buffer := make([]byte, 100) _, err = file.Read(buffer) if err != nil { fmt.Println("Error reading file:", err) return } fmt.Println("Read:", string(buffer)) } """ ### 2.6 Testing **Standard:** Write unit tests, integration tests, and end-to-end tests to ensure code quality and prevent regressions. **Why:** Testing is crucial for verifying the correctness of the code and preventing bugs from being introduced. **Do This:** * Write unit tests for individual components and functions. * Write integration tests to verify that components work together correctly. * Write end-to-end tests to verify that the entire system works as expected. * Use a testing framework (e.g., "testing" package in Go). * Automate the execution of tests. * Aim for high test coverage. **Don't Do This:** * Skip writing tests. * Write tests that are too brittle (i.e., that break easily when the code is changed). * Write tests that are too slow to execute. ## 3. Ollama-Specific Considerations ### 3.1 Modelfiles When working with modelfiles, treat them as configuration as code. Changes should be automated via tooling, not manual edits. ### 3.2 Interfacing with External Libraries Ollama often integrates with external libraries for tasks like model loading, inference, and data processing. It's important to handle these integrations carefully. **Standard:** * Use clear and well-defined interfaces to abstract the interaction with external libraries. * Implement error handling to gracefully handle failures in external libraries. * Consider using dependency injection to make it easier to swap out different implementations of external libraries. **Example:** If using a C++ library for model inference, create a Go package that wraps the C++ library and provides a Go-friendly interface. Handle any errors that occur in the C++ library and translate them into Go errors. ### 3.3 Performance Optimization Ollama is often used for computationally intensive tasks. Performance is a critical consideration. **Standard:** * Profile the code to identify performance bottlenecks. * Use efficient algorithms and data structures. * Optimize memory usage. * Use concurrency to take advantage of multi-core processors. * Consider using caching to reduce the load on external resources. **Example:** When performing model inference, consider using batching to process multiple inputs at once. This can improve throughput by amortizing the overhead of model loading and inference. Use tools like "pprof" to diagnose performance issues. ### 3.4 Security Ollama may handle sensitive data. Security is a paramount concern. **Standard:** * Follow security best practices for all code. * Validate all inputs to prevent injection attacks. * Protect sensitive data (e.g., API keys, passwords) using encryption. * Regularly audit the code for security vulnerabilities. * Be aware of common LLM security vulnerabilities such as prompt injection, and implement safeguards. **Example:** When handling user input, sanitize the input to prevent cross-site scripting (XSS) and other injection attacks. Use a secure storage mechanism to store API keys and passwords. ## 4. Modern Approaches and Patterns Ollama should strive to incorporate modern software development principles and well-established design patterns. ### 4.1 Dependency Injection As shown in prior examples, use dependency injection to decouple components. ### 4.2 Functional Programming Where appropriate leverage functional programming concepts such as pure functions, immutability, and higher-order functions to improve code clarity and testability. ### 4.3 Observability Implement robust monitoring and tracing to gain insights into the performance and behavior of Ollama components in production. ### 4.4 Asynchronous Programming Employ asynchronous programming techniques such as goroutines and channels to handle concurrent requests efficiently and prevent blocking operations. ### 4.5 Immutability Ensure that once an object is created, its state cannot be changed. This helps to avoid race conditions and makes it easier to reason about the code. ## 5. Anti-Patterns and Common Mistakes ### 5.1 Premature Optimization Do not optimize code before profiling and identifying actual performance bottlenecks. ### 5.2 Reinventing the Wheel Leverage existing libraries and frameworks instead of implementing functionality from scratch. ### 5.3 Ignoring Errors As previously mentioned, ignoring errors can lead to unpredictable behavior and make it difficult to debug issues. ### 5.4 Over-Engineering Avoid creating overly complex solutions when simpler alternatives exist. This can increase the maintenance burden and make the code harder to understand. ### 5.5 God Classes Avoid creating classes that have too many responsibilities, as this can make the code harder to maintain and test. By following these component design standards, Ollama developers can create a codebase that is robust, maintainable, and performant. This will lead to a better user experience and a more successful project.