Cline

# Testing Methodologies Standards for Ollama This document outlines the testing methodologies standards for contributing to the Ollama project. Adhering to these standards ensures code quality, maintainability, reliability, and security. These guidelines should be used by developers and AI coding assistants alike. ## 1. Overview of Testing in Ollama Testing in Ollama is crucial for ensuring the stability and reliability of the platform. Given Ollama's role in serving and managing large language models, robust testing is essential to guarantee correct model serving, efficient resource utilization, and secure operation. This document outlines unit, integration, and end-to-end testing approaches tailored for Ollama. ### 1.1. Test Pyramid We follow the test pyramid concept, emphasizing a large number of unit tests, a reasonable amount of integration tests, and fewer end-to-end tests. This balance helps provide comprehensive coverage while minimizing test execution time and maximizing isolation. ### 1.2. Test Driven Development (TDD) While not strictly enforced, Test Driven Development (TDD) is encouraged, especially for new features. Writing tests before implementation helps clarify requirements and ensures that code is testable from the start. ## 2. Unit Testing Standards Unit tests should focus on testing individual components or functions in isolation. They should be fast, reliable, and easy to maintain. In Ollama, these tests are particularly important for the core components like model loading, inference engines, and API handlers. ### 2.1. Guidelines * **Do This**: Aim for high code coverage (ideally > 80%) for all core components. * **Do This**: Mock external dependencies (e.g., file system access, network calls) to isolate the unit under test. * **Do This**: Write clear and descriptive test names that accurately reflect the behavior being tested. * **Do This**: Use assertion libraries to verify expected outcomes. * **Don't Do This**: Perform I/O operations or rely on external services in unit tests. * **Don't Do This**: Write overly complex tests that are difficult to understand or maintain. * **Don't Do This**: Neglect edge cases and boundary conditions. * **Why**: Unit tests provide the fastest feedback loop and are essential for catching regressions early. ### 2.2. Specific Implementations & Examples Assume Ollama uses Go as its primary language. """go // Example: Unit test for a function that loads a model configuration package model import ( "testing" "os" ) func TestLoadModelConfig(t *testing.T) { // Create a temporary file with a sample model config tempFile, err := os.CreateTemp("", "model_config.yaml") if err != nil { t.Fatalf("Failed to create temporary file: %v", err) } defer os.Remove(tempFile.Name()) // Clean up after the test defer tempFile.Close() _, err = tempFile.WriteString(" name: my_model format: llama params: temperature: 0.7 top_p: 0.9 ") if err != nil { t.Fatalf("Failed to write to temporary file: %v", err) } //reset file pointer tempFile.Seek(0,0) // Call the function being tested (loadModelConfig) config, err := loadModelConfig(tempFile.Name()) // Assuming loadModelConfig takes the filename // Assert that there is no error and the config is loaded correctly if err != nil { t.Fatalf("Failed to load model config: %v", err) } if config.Name != "my_model" { t.Errorf("Expected model name 'my_model', got '%s'", config.Name) } if config.Format != "llama"{ t.Errorf("Expected model format 'llama', got '%s'", config.Format) } if config.Params["temperature"] != 0.7 { t.Errorf("Expected temperature 0.7, got %f", config.Params["temperature"]) } } //Example mocking function func loadModelConfig(filename string) (*ModelConfig, error) { //implementation accessing file system return &ModelConfig{ Name: "test_model", Format: "test_format", Params: map[string]interface{}{ "temperature": 0.5, }, }, nil } type ModelConfig struct { Name string Format string Params map[string]interface{} } """ **Explanation:** * The test creates a temporary file containing a sample model configuration. * It then calls the "loadModelConfig" and asserts that it returns no error and that the loaded configuration has the expected values. * The use of "os.CreateTemp" and "os.Remove" ensures that the test doesn't interfere with the actual filesystem and the temporary file is deleted after the test. * This example also uses t.Fatalf, which will terminate the test immediately allowing for quick debugging and error detection. * The example "loadModelConfig" is simplified for demonstration using hardcoded responses. In reality, this should parse and validate the YAML from the temporary file that was created. **Anti-Pattern:** """go // Anti-pattern: Unit test that performs I/O func TestLoadModelConfig_InvalidFile(t *testing.T) { // This test attempts to read a file directly, which is not a good practice // because it depends on the external environment. _, err := loadModelConfig("/path/to/nonexistent/file.yaml") // Dangerous I/O operation if err == nil { t.Errorf("Expected an error, but got nil") } } """ Instead, mock the loadModelConfig function to simulate a file not found error. ### 2.3. Mocking Libraries Use mocking libraries (like "testify/mock" in Go) to create mock implementations of interfaces and dependencies. This allows you to control the behavior of external components for testing purposes. """go // Example: Using testify/mock to mock an external service package ollama import ( "testing" "github.com/stretchr/testify/mock" ) // Define an interface that an external service implements type ModelLoader interface { LoadModel(modelName string) (string, error) } // Create a mock implementation of the ModelLoader interface type MockModelLoader struct { mock.Mock } // Implement the LoadModel method for the mock func (m *MockModelLoader) LoadModel(modelName string) (string, error) { args := m.Called(modelName) return args.String(0), args.Error(1) } // Function to test that uses the ModelLoader interface func UseModelLoader(loader ModelLoader, modelName string) (string, error) { return loader.LoadModel(modelName) } func TestUseModelLoader(t *testing.T) { // Create a new mock ModelLoader instance mockLoader := new(MockModelLoader) // Define what the mock should return when LoadModel is called with "test_model" mockLoader.On("LoadModel", "test_model").Return("model_content", nil) // Use the mock in our test result, err := UseModelLoader(mockLoader, "test_model") // Assert that the result and error are as expected if err != nil { t.Fatalf("Unexpected error: %v", err) } if result != "model_content" { t.Errorf("Expected 'model_content', but got '%s'", result) } // Assert that the mock's LoadModel method was called once with the correct argument mockLoader.AssertExpectations(t) } """ **Explanation:** * The MockModelLoader allows for controlling the exact outputs of the LoadModel, while asserting that the function was only called once. * This ensures only the function UseModelLoader is tested, and the outputs can be reliably checked. ## 3. Integration Testing Standards Integration tests verify the interaction between different components or modules of the Ollama system. These tests are more comprehensive than unit tests and help uncover issues related to component dependencies and communication. Specifically, focus on testing the integration of model loading with inference engines, API endpoints with authentication mechanisms, and various data storage components. ### 3.1. Guidelines * **Do This**: Test the integration of two or more components, focusing on data flow and interactions. * **Do This**: Use real dependencies where appropriate, but consider using test containers or lightweight substitutes to avoid external service dependencies. * **Do This**: Ensure that integration tests cover common use cases and error scenarios. * **Don't Do This**: Mock internal components excessively; the goal is to test real interactions. * **Don't Do This**: Write integration tests that are too broad and test the entire system; focus on specific integrations. * **Why**: Integration tests catch issues that unit tests might miss, such as incorrect data mapping or communication protocol errors. ### 3.2. Specific Implementations & Examples """go // Example: Integration test for loading a model and running a simple inference package ollama import ( "testing" // "ollama.ai/ollama/pkg/llm" // Use the actual llm package "os" "fmt" ) // Mock llm interface for demonstrating the integration test type LLM interface { Predict(prompt string) (string, error) } type MockLLM struct {} func (m *MockLLM) Predict(prompt string) (string, error) { return fmt.Sprintf("Response for prompt: %s", prompt), nil } //Simplified implementation that does not actually load a model func loadAndPredict(modelPath string, prompt string) (string, error) { llm := &MockLLM{} return llm.Predict(prompt) //Simlified to llm.Predict for the mock } func TestLoadModelAndPredict(t *testing.T) { // Create a model directory and a dummy model file modelDir := t.TempDir() modelFile := modelDir + "/test_model.bin" err := os.WriteFile(modelFile, []byte("dummy model content"), 0644) //Creating a dummy file instead of loading a real model if err != nil { t.Fatalf("Failed to create model file: %v", err) } // Define a test prompt prompt := "What is the capital of France?" // Call the integration function result, err := loadAndPredict(modelFile, prompt) // Assert that the result is as expected if err != nil { t.Fatalf("Unexpected error: %v", err) } expectedResult := "Response for prompt: What is the capital of France?" if result != expectedResult { t.Errorf("Expected '%s', but got '%s'", expectedResult, result) } } """ **Explanation:** * The test creates a temporary directory and a dummy model file to simulate a real model. (In a real implemntation, this would load an actual model using the Ollama libraries.) * It then calls the "loadAndPredict" function, which simulates loading the model and running a prediction. The mock avoids external calls for the sake of testing. * The test asserts that the result is as expected. * This example checks the integration between the model loading and inference components. **Anti-Pattern:** """go // Anti-pattern: Integration test that is too broad func TestEntireOllamaSystem(t *testing.T) { // This test attempts to test the entire Ollama system, which is not a good practice // because it is too broad and difficult to maintain. It is brittle and will break often when the backend implementation changes. // Instead, focus on specific integrations. // ... test logic that covers many modules ... } """ ## 4. End-to-End (E2E) Testing Standards End-to-end tests simulate real user interactions with the Ollama system. These tests verify that the entire system works correctly, from the user interface to the backend services. These tests validate critical workflows, such as deploying a new model, querying it via the API, and monitoring its performance. ### 4.1. Guidelines * **Do This**: Focus on critical user journeys and high-level scenarios. * **Do This**: Use a dedicated test environment that closely mirrors the production environment. * **Do This**: Automate E2E tests to run regularly as part of the CI/CD pipeline. * **Don't Do This**: Write E2E tests that are too granular or cover minor details; focus on the big picture. * **Don't Do This**: Neglect performance and security testing in E2E scenarios. * **Why**: E2E tests provide the highest level of confidence that the system works as expected in real-world scenarios. ### 4.2. Specific Implementations & Examples """go // Example: End-to-end test for deploying and querying a model package ollama import ( "testing" "os/exec" "net/http" "io/ioutil" "strings" "time" ) // Function to check if the Ollama server is running func isOllamaRunning() bool { resp, err := http.Get("http://localhost:11434/") // Assuming default port if err != nil { return false } defer resp.Body.Close() return resp.StatusCode == http.StatusOK } // Function to run Ollama commands func runOllamaCommand(command string) (string, error) { cmd := exec.Command("ollama", strings.Split(command, " ")...) out, err := cmd.CombinedOutput() return string(out), err } func TestDeployAndQueryModel(t *testing.T) { // Ensure Ollama server is running if !isOllamaRunning() { t.Skip("Ollama server is not running; skipping E2E test") } // 1. Deploy a model modelName := "test_model" _, err := runOllamaCommand("create " + modelName + " FROM 'FROM scratch\nCMD echo \'Hello, world!\''") //Simplified Modelfile if err != nil { t.Fatalf("Failed to deploy model: %v", err) } defer runOllamaCommand("rm " + modelName) // Cleanup after the test // 2. Give the server time to load the model time.Sleep(5 * time.Second) // 3. Query the model resp, err := http.Post("http://localhost:11434/api/generate", "application/json", strings.NewReader("{"prompt": "Say hello"}")) if err != nil { t.Fatalf("Failed to query model: %v", err) } defer resp.Body.Close() body, err := ioutil.ReadAll(resp.Body) if err != nil { t.Fatalf("Failed to read response body: %v", err) } // 4. Assert that the response is as expected expectedResponse := "Hello, world!" if !strings.Contains(string(body), expectedResponse) { t.Errorf("Expected response to contain '%s', but got '%s'", expectedResponse, string(body)) } } """ **Explanation:** * The test first checks if the Ollama server is running. * It then deploys a test model using the "ollama create" command with a simplified Modelfile. * It gives the server some time to load the model. * It then queries the model via the API and asserts that the response contains the expected content. * The test uses basic command execution and HTTP requests to simulate real user interactions. * The "defer runOllamaCommand("rm " + modelName)" ensures that the test cleans up after itself by removing the deployed model. **Anti-Pattern:** """go // Anti-pattern: E2E test that relies on specific UI elements func TestUIInteraction(t *testing.T) { // This test attempts to interact with specific UI elements, which is not a good practice // because it is brittle and will break easily when the UI changes. // Instead, focus on critical user journeys and high-level scenarios. // ... test logic that depends on specific UI elements ... } """ ### 4.3. Infrastructure and Tools * Utilize tools like Docker Compose to manage test environments with multiple Ollama instances and dependencies. * Consider using dedicated E2E testing frameworks like Cypress or Playwright for browser-based testing if Ollama has a web UI component. ## 5. Performance Testing Standards Performance testing is crucial for ensuring that Ollama can handle the required load and provide acceptable response times. These tests should measure metrics such as requests per second, latency, and resource utilization. ### 5.1. Guidelines * **Do This**: Define clear performance goals and metrics based on expected usage patterns. * **Do This**: Use load testing tools (e.g., Apache JMeter, k6) to simulate realistic traffic. * **Do This**: Monitor resource utilization (CPU, memory, network) during performance tests. * **Do This**: Identify and address performance bottlenecks through profiling and optimization. * **Don't Do This**: Perform performance tests in isolation; consider the impact of other services and dependencies. * **Don't Do This**: Neglect long-running tests to identify memory leaks or other stability issues. * **Why**: Performance tests ensure that Ollama can meet the performance requirements of its users and prevent performance degradation over time. ### 5.2. Specific Implementations & Examples """go // Example: Performance test using Apache JMeter // 1. Create a JMeter test plan (e.g., ollama_performance_test.jmx) that simulates // multiple users querying the Ollama API concurrently. The test plan should define // a range of variables, like thread count, ramp-up, and number of loops. // 2. Configure the JMeter test plan to send HTTP requests to the Ollama API endpoint // (e.g., http://localhost:11434/api/generate) with different prompts. // 3. Run the JMeter test plan from the command line: // jmeter -n -t ollama_performance_test.jmx -l ollama_performance_results.jtl // 4. Analyze the results in the ollama_performance_results.jtl file to identify // performance bottlenecks and areas for optimization. Use JMeter's reporting tools. //Example Code (Go) to measure a model's inference time. package ollama import ( "testing" "time" ) func benchmarkModelInference(b *testing.B, prompt string) { llm := &MockLLM{} //Using the mock LLM created in previous section b.ResetTimer() for i := 0; i < b.N; i++ { _, _ = llm.Predict(prompt) } } func BenchmarkModelInferenceShortPrompt(b *testing.B) { benchmarkModelInference(b, "Short prompt") } func BenchmarkModelInferenceLongPrompt(b *testing.B) { longPrompt := strings.Repeat("This is a long prompt. ", 100) benchmarkModelInference(b,longPrompt) } """ **Explanation:** * Benchmark tests are created for short and long prompts. These tests use the mock llm predict function to benchmark the function. * When running the "go test -bench=." command, go will run these benchmarks and provide information on performance. ## 6. Security Testing Standards Security testing is essential to identifying and mitigating security vulnerabilities in the Ollama system. These tests should cover areas such as authentication, authorization, input validation, and data protection. ### 6.1. Guidelines * **Do This**: Perform regular security audits and penetration tests to identify potential vulnerabilities. * **Do This**: Implement robust authentication and authorization mechanisms to protect access to sensitive resources. * **Do This**: Validate all user inputs to prevent injection attacks and other security exploits. * **Do This**: Encrypt sensitive data both in transit and at rest. * **Don't Do This**: Store sensitive information (e.g., API keys, passwords) in plaintext. * **Don't Do This**: Neglect security considerations during the design and development phases. * **Why**: Security tests protect the Ollama system from unauthorized access, data breaches, and other security threats. ### 6.2. Specific Implementations & Examples * Use tools like OWASP ZAP or Burp Suite to perform dynamic application security testing (DAST). * Implement static analysis tools (e.g., SonarQube, GoSec) to identify security vulnerabilities in the code. * Perform manual code reviews to identify security flaws that automated tools might miss. **Specific Ollama examples will depend on the final architecture designed.** As such, design guidance can be provided: * Authentication Testing: Test authentication failures (e.g. brute forcing login, bypassing authentication mechanisms, invalid or expired tokens), Authorization testing (e.g. privilege escalation, access to resources for unauthorized users), and testing the retrieval of authorized resources only. * Implement testing for input validation: Perform fuzzing on API parameters with unexpected values to test if the server handles them correctly, and test for injection attacks. Examine the API for various injection types (SQL, command). * Rate Limiting: Ensure rate limiting middleware is properly configured and prevents abuse, with tests that call rapid, repeated API requests to look for throttling. ## 7. Testing in CI/CD Pipeline Automate all types of tests (unit, integration, E2E, performance, security) in the CI/CD pipeline to ensure continuous feedback and prevent regressions from reaching production. ### 7.1. Guidelines * **Do This**: Integrate tests into the CI/CD pipeline to run automatically on every code change. * **Do This**: Use a dedicated testing environment in the CI/CD pipeline. * **Do This**: Fail the build if any tests fail. * **Do This**: Provide clear and actionable feedback to developers when tests fail. * **Why**: Automated testing in the CI/CD pipeline ensures that code changes are thoroughly tested before being deployed to production. ### 7.2. Specific Implementations & Examples * Use CI/CD tools (e.g., Jenkins, GitLab CI, GitHub Actions) to define test pipelines. * Configure the CI/CD pipeline to run unit tests, integration tests, and E2E tests in parallel to reduce build time. * Integrate performance and security testing into the CI/CD pipeline to identify performance regressions and security vulnerabilities early in the development process.

# Tooling and Ecosystem Standards for Ollama This document outlines the coding standards for the Ollama project related to tooling and the broader ecosystem. Following these guidelines will ensure a consistent, maintainable, and performant codebase across all Ollama components and integrations. These standards are designed to be used by both human developers and AI coding assistants. ## 1. Development Environment Setup ### Standard 1.1: Consistent Development Environment **Do This:** * Use a consistent and reproducible development environment. Define this via Docker, Nix, or similar tools to guarantee that developers have the same dependencies and versions. * Employ a "devcontainer.json" file for VS Code or similar configuration files for other IDEs like IntelliJ or other editors. This facilitates automated environment setup. **Don't Do This:** * Rely on manually installed dependencies. This creates inconsistencies and "works on my machine" problems. **Why:** A consistent environment minimizes integration issues and reduces the debugging time associated with environment-specific bugs. **Example ("devcontainer.json"):** """json { "name": "Ollama Dev Environment", "build": { "dockerfile": "Dockerfile.dev" }, "extensions": [ "golang.go", "ms-azuretools.vscode-docker" ], "settings": { "go.toolsManagement.checkForUpdates": "local", "go.useLanguageServer": true, "go.gopath": "/go" }, "mounts": [ "source=${localWorkspaceFolder},target=/go/src/github.com/ollama/ollama,type=bind,consistency=cached" ], "runArgs": [ "--security-opt", "apparmor=unconfined" ] } """ **Dockerfile.dev:** """dockerfile FROM golang:1.21-alpine AS builder WORKDIR /app COPY go.mod go.sum ./ RUN go mod download COPY . . RUN go build -o ollama ./cmd/ollama FROM alpine:latest WORKDIR /app COPY --from=builder /app/ollama . # Copy necessary binaries and scripts used by ollama (example) COPY --from=builder /app/scripts ./scripts EXPOSE 11434 CMD ["./ollama", "serve"] """ **Anti-Pattern:** Omitting a "devcontainer.json" or similar setup, leading to developers spending excessive time configuring their environments. ### Standard 1.2: Dependency Management **Do This:** * Use Go modules (or the project’s established dependency management tool) to manage dependencies. * Vendoring dependencies when appropriate to ensure reproducibility and isolation from external changes. * Regularly update dependencies to receive security patches and bug fixes, but always test after updating! **Don't Do This:** * Rely on globally installed packages or outdated dependency management techniques. * Ignore dependency updates or skip security vulnerability checks. **Why:** Robust dependency management prevents version conflicts and ensures that the project remains stable and secure. **Example:** """bash go mod tidy go mod vendor """ **Anti-Pattern:** Not running "go mod tidy" or "go mod vendor" before committing code. ## 2. Code Analysis and Formatting ### Standard 2.1: Automated Code Formatting **Do This:** * Use "gofmt" or "goimports" for automatic code formatting. Integrate it into your IDE and CI/CD pipeline. * Configure your editor to format code on save. **Don't Do This:** * Manually format code or ignore formatting errors. * Disable code formatting tools. **Why:** Consistent code formatting enhances readability and reduces the cognitive load when reviewing code. **Example:** """bash go fmt ./... goimports -w ./... """ **Configuration Example (VS Code):** """json { "go.formatTool": "goimports", "editor.formatOnSave": true } """ **Anti-Pattern:** Inconsistent indentation and spacing, making the code difficult to read. ### Standard 2.2: Linting and Static Analysis **Do This:** * Use "golangci-lint" with a predefined configuration file (".golangci.yml"). * Run linters as part of your CI/CD pipeline. * Address linting issues promptly. **Don't Do This:** * Ignore linting warnings or errors. * Disable linters without a valid reason. **Why:** Linting helps identify potential bugs, code smells, and style inconsistencies. **Example (".golangci.yml"):** """yaml run: deadline: 5m tests: false linters: enable: - gofmt - goimports - govet - revive - staticcheck - unconvert - misspell - unparam - unused """ **Running Linter:** """bash golangci-lint run ./... """ **Anti-Pattern:** Committing code with linting errors, indicating a lack of attention to detail. ## 3. Testing and Quality Assurance ### Standard 3.1: Comprehensive Unit Tests **Do This:** * Write unit tests for all non-trivial functions and methods. * Aim for high code coverage (e.g., >80%). Use coverage reports to identify gaps in testing. * Use table-driven tests for parameterized testing. **Don't Do This:** * Skip unit tests for complex logic. * Have low code coverage. * Write trivial tests that don't properly validate functionality. **Why:** Unit tests verify the correctness of individual components and prevent regressions. **Example:** """go package mypackage import "testing" func TestAdd(t *testing.T) { testCases := []struct { a, b, expected int }{ {1, 2, 3}, {0, 0, 0}, {-1, 1, 0}, } for _, tc := range testCases { result := Add(tc.a, tc.b) if result != tc.expected { t.Errorf("Add(%d, %d) = %d, expected %d", tc.a, tc.b, result, tc.expected) } } } """ **Anti-Pattern:** Writing tests that only check for error conditions and not the correct functionality. ### Standard 3.2: Integration Tests **Do This:** * Write integration tests to verify the interaction between different components of the system and external services (e.g., Ollama APIs, databases, etc.). * Use mocking frameworks to isolate components during testing. * Include tests that cover common use cases and error scenarios. **Don't Do This:** * Skip integration tests and assume components will work together correctly. * Rely solely on unit tests. **Why:** Integration tests ensure that the system functions correctly as a whole. **Example (using a mocking framework, conceptually):** """go package integrationtest import ( "testing" "net/http" //Example import "github.com/golang/mock/gomock" "github.com/ollama/ollama/api" // Hypothetical Ollama API package ) func TestMyIntegration(t *testing.T) { ctrl := gomock.NewController(t) defer ctrl.Finish() // Mock the Ollama API client //mockOllamaClient := api.NewMockOllamaClient(ctrl) //Example mocking and expectation //mockOllamaClient.EXPECT().PullModel(gomock.Any(), "my-model").Return(nil) // Your integration test logic here, using the mockOllamaClient // ... test code utilizing the Mock } """ **Anti-Pattern:** Integration tests that are too brittle and fail based on minor configuration changes. Avoid hardcoding IPs or other environment-specific settings. ### Standard 3.3: Performance and Load Testing **Do This:** * Conduct performance and load testing to identify potential bottlenecks and performance issues. * Use tools like "k6" or "vegeta" to simulate realistic workloads. * Monitor system metrics (e.g., CPU usage, memory usage, latency) during testing. * Integrate performance tests into CI/CD to detect performance regressions early. **Don't Do This:** * Ignore performance testing until late in the development cycle. * Fail to monitor system performance during testing. **Why:** Performance testing helps ensure that the system can handle the expected load and meet performance requirements. **Example (k6 script):** """javascript import http from 'k6/http'; import { check } from 'k6'; export const options = { vus: 10, duration: '10s', }; export default function () { const res = http.get('http://localhost:11434/api/tags'); check(res, { 'status is 200': (r) => r.status === 200, }); } """ **Running k6:** """bash k6 run script.js """ **Anti-Pattern:** Failing to establish baseline performance metrics and track performance changes over time. ## 4. Documentation ### Standard 4.1: Godoc Comments **Do This:** * Write clear and concise Godoc comments for all exported types, functions, and methods. * Include examples in your Godoc comments to illustrate how to use the code. * Keep Godoc comments up-to-date as the code evolves. **Don't Do This:** * Omit Godoc comments for exported APIs. * Write vague or misleading comments. * Let Godoc comments become stale. **Why:** Godoc comments provide essential documentation for users of the Ollama API. **Example:** """go // Add adds two integers and returns the result. // // Example: // // result := Add(1, 2) // fmt.Println(result) // Output: 3 func Add(a, b int) int { return a + b } """ **Anti-Pattern:** Using overly verbose comments that repeat what the code already expresses. ### Standard 4.2: Architecture and Design Documents. **Do This:** * Maintain high-level architecture and design documents, ideally in a tool like C4 or similar, to clearly outline components in Ollama and their integration. * Update these documents whenever there are significant changes to the system's architecture. Store them near the code! **Don't Do This:** * Let architecture documents become outdated * Fail to document key design decisions **Why:** Architecture and Design Documents ensures all developers have a clear overview of the project as a whole. ## 5. Logging and Monitoring ### Standard 5.1: Structured Logging **Do This:** * Use structured logging (e.g., with "zap" or "logrus") to log events in a consistent and machine-readable format. * Include relevant context in your logs (e.g., request ID, user ID, component name). * Log at appropriate levels (e.g., debug, info, warn, error, fatal). * Use log correlation IDs to track requests across multiple services. **Don't Do This:** * Use plain text logging, which is difficult to parse and analyze. * Log sensitive information (e.g., passwords, API keys). * Over-log or under-log events. **Why:** Structured logging simplifies log analysis and debugging. **Example (using "zap"):** """go package main import ( "go.uber.org/zap" ) func main() { logger, _ := zap.NewProduction() defer logger.Sync() // flushes buffer, if any sugar := logger.Sugar() sugar.Infow("failed to fetch URL", "url", "http://example.com", "attempt", 3, "err", err, ) } """ **Anti-Pattern:** Logging error messages without providing sufficient context to diagnose the problem. ### Standard 5.2: Application Monitoring **Do This:** * Use a monitoring system (e.g., Prometheus, Grafana) to track key metrics (e.g., request latency, error rates, resource usage). * Set up alerts for critical events (e.g., high error rates, resource exhaustion). * Use distributed tracing (e.g., Jaeger, Zipkin) to trace requests across multiple services. **Don't Do This:** * Fail to monitor application performance. * Ignore alerts. **Why:** Monitoring helps detect and diagnose problems promptly. **Example (Prometheus integration):** """go package main import ( "net/http" "github.com/prometheus/client_golang/prometheus/promhttp" ) func main() { http.Handle("/metrics", promhttp.Handler()) http.ListenAndServe(":2112", nil) } """ **Anti-Pattern:** Alerting on metrics that trigger too frequently and desensitize the team, or conversely setting thresholds too high so real issues are missed. ## 6. Security ### Standard 6.1: Secure Coding Practices **Do This:** * Follow secure coding practices to prevent common vulnerabilities (e.g., SQL injection, XSS, CSRF). * Validate all input data. * Use parameterized queries to prevent SQL injection. * Encode output data to prevent XSS. * Implement CSRF protection. **Don't Do This:** * Trust user input without validation. * Use dynamic SQL queries directly from user input. **Why:** Security is paramount to the integrity and trustworthiness of Ollama. ### Standard 6.2: Vulnerability Scanning **Do This:** * Use vulnerability scanning tools (e.g., "govulncheck") to identify security vulnerabilities in dependencies. * Regularly update dependencies to patch known vulnerabilities. **Don't Do This:** * Ignore vulnerability scans. * Fail to update dependencies with known vulnerabilities. **Why:** Proactive vulnerability scanning helps prevent security breaches. **Example:** """bash go install golang.org/x/vuln/cmd/govulncheck@latest govulncheck ./... """ **Anti-Pattern:** Ignoring vulnerability reports and accumulating vulnerable dependencies. ## 7. Ollama Specific Ecosystem Considerations ### Standard 7.1: Utilize Official Ollama Libraries and Tools **Do This:** * Prefer official Ollama libraries and tools when available. * Adhere to the documented APIs and usage patterns. * Contribute back to the official libraries when possible. **Don't Do This:** * Reinvent the wheel with custom solutions when official libraries exist. * Use undocumented features or APIs that may change without notice. **Why:** Using official libraries ensures compatibility and leverages the expertise of the Ollama project. **Example:** When interacting with the Ollama API, use the official Go client library (if it exists). If performing file operations on Ollama's data directory, use the provided utility functions, rather than directly manipulating files. ### Standard 7.2: Modelfile Best Practices **Do This:** * Follow the Modelfile documentation when creating custom models. * Use descriptive comments within the Modelfile to explain the model's configuration. * Test Modelfiles thoroughly to ensure they produce the desired results. * Use variables within the Modelfile for parameters, to make it more readable. * Consider using stages within the Modelfile to reduce the size of the final model by removing intermediate files. **Don't Do This:** * Use magic numbers or unclear configuration settings in your Modelfiles. * Create excessively large or complex Modelfiles without proper documentation. * Store sensitive data (e.g., API keys) directly in Modelfiles. **Why:** Well-written Modelfiles contribute to model reproducibility and ease of use. **Example:** """dockerfile # Modelfile for a CodeLlama model fine-tuned for Python code generation. FROM codellama # Set the temperature for code generation. PARAMETER temperature 0.7 # Install necessary Python libraries. RUN pip install numpy pandas requests # Add custom code generation scripts. COPY scripts/ /app/scripts/ # Run the code generation script. CMD ["python", "/app/scripts/generate_code.py"] """ ### Standard 7.3: Community Engagement **Do This:** * Engage with the Ollama community through forums, chat channels, and issue trackers. * Share your experiences and contribute to the project's knowledge base. * Provide feedback on existing tools and libraries. **Don't Do This:** * Be afraid to ask questions or seek help from the community. * Ignore community feedback on your code or contributions. **Why:** Community engagement fosters collaboration and accelerates the development of the Ollama ecosystem. By adhering to these coding standards, we can ensure that the Ollama project remains maintainable, performant, and secure, while also fostering a thriving ecosystem of tools and libraries.

# Code Style and Conventions Standards for Ollama This document outlines the coding style and conventions to be followed when contributing to the Ollama project. Adhering to these guidelines ensures code consistency, readability, and maintainability across the project. These standards are intended to guide developers, act as a reference for code reviews, and offer context for AI coding assistants. ## 1. General Principles ### 1.1. Consistency * **Do This:** Maintain consistency within a file, module, and across the entire project. Adopt existing patterns and styles. * **Don't Do This:** Introduce new and inconsistent styles without prior discussion and agreement. * **Rationale:** Inconsistent code reduces readability and increases cognitive load. By adhering to a uniform styling, the project becomes more accessible and easier to maintain. ### 1.2. Readability * **Do This:** Write code that is easy to understand at a glance. Use meaningful names, comments, and spacing. * **Don't Do This:** Write complex or obfuscated code that requires significant effort to understand. * **Rationale:** Readable code allows other developers (and your future self) to quickly grasp the logic and intent, reducing maintenance time and bugs introduced during modifications. ### 1.3. Simplicity * **Do This:** Aim for the simplest solution that meets the requirements. Avoid over-engineering. * **Don't Do This:** Introduce unnecessary complexity or abstractions. * **Rationale:** Simpler code is easier to test, debug, and maintain. It also reduces the likelihood of introducing bugs. ### 1.4. Clarity * **Do This:** Ensure the purpose of each code block (function, class, or module) is clear and easily understandable. Use comments to explain complex logic or non-obvious decisions. * **Don't Do This:** Leave segments of code without clearly defined purpose or without adequate documentation, forcing other developers to reverse engineer the intent. * **Rationale:** Clarity is crucial for collaboration. When the purpose and function of the code are immediately obvious the barrier to contribution is dramatically lowered thereby increasing collaboration and reducing debugging effort. ## 2. Language-Specific Conventions (Go) Ollama is primarily written in Go. Therefore, Go-specific conventions are very important. ### 2.1. Formatting * **Do This:** Use "gofmt" to automatically format your code. Configure your editor to run "gofmt" on save. * **Don't Do This:** Manually format code or ignore "gofmt" suggestions. * **Rationale:** "gofmt" enforces a single, consistent style across the codebase, eliminating debates about formatting. """go // Example of properly formatted Go code package main import "fmt" func main() { message := "Hello, Ollama!" fmt.Println(message) } """ ### 2.2. Naming Conventions * **Do This:** * Use "CamelCase" for exported types and functions. * Use "camelCase" for unexported types and functions. * Use meaningful and descriptive names. * Use acronyms in all caps (e.g., "APIURL", "HTTPClient"). * Follow Go's naming conventions for variables (e.g., "i" for loop counters, "err" for errors). * **Don't Do This:** * Use abbreviations that are not widely understood. * Use names that are too short or too long. * Use inconsistent naming styles. * Prefix variable names with types (Hungarian notation). * **Rationale:** Consistent naming improves readability and helps understand the purpose and scope of variables, types, and functions. """go // Example of good naming conventions type APIClient struct { APIURL string } func (c *APIClient) FetchData(resourceID string) ([]byte, error) { // ... } // Example of function with good meaningful name func processUserInput(inputString string) (string, error) { // ... } """ ### 2.3. Comments and Documentation * **Do This:** * Write clear and concise comments to explain complex logic or non-obvious decisions. * Document all exported types, functions, and constants using Go's documentation comments. * Use sentence-case for comments, starting with a capital letter and ending with a period. * **Don't Do This:** * Write comments that state the obvious. * Leave code undocumented. * Use informal language or abbreviations in comments. * **Rationale:** Good documentation is essential for understanding and maintaining the codebase. "godoc" extracts these comments to generate API documentation. Clarity regarding purpose and usage of the function helps other developers to quickly start using it. """go // APIClient is a client for interacting with the Ollama API. type APIClient struct { APIURL string // APIURL is the URL of the Ollama API. } // FetchData retrieves data from the Ollama API for the given resource ID. func (c *APIClient) FetchData(resourceID string) ([]byte, error) { // ... } """ ### 2.4. Error Handling * **Do This:** * Always check for errors. * Return errors as the last return value. * Use "errors.Is" and "errors.As" from the "errors" package (or "xerrors" for earlier Go versions) to check for specific errors. * Provide context to errors using "fmt.Errorf" or the "errors.Wrap" function (from "pkg/errors" if needed). * Log errors with sufficient context. * **Don't Do This:** * Ignore errors. * Panic unless absolutely necessary (e.g., unrecoverable initialization errors). * Use naked returns without considering the error value. * **Rationale:** Proper error handling prevents unexpected program behavior and makes debugging easier. Including context in error messages helps pinpoint the location and cause of errors. """go func fetchData(url string) ([]byte, error) { resp, err := http.Get(url) if err != nil { return nil, fmt.Errorf("failed to fetch data from %s: %w", url, err) } defer resp.Body.Close() if resp.StatusCode != http.StatusOK { return nil, fmt.Errorf("unexpected status code %d", resp.StatusCode) } body, err := io.ReadAll(resp.Body) if err != nil { return nil, fmt.Errorf("failed to read response body: %w", err) } return body, nil } func main() { data, err := fetchData("https://example.com/api/data") if err != nil { log.Printf("Error: %v", err) return } fmt.Println(string(data)) } """ ### 2.5. Concurrency * **Do This:** * Use goroutines and channels for concurrent operations. * Use "sync" package primitives to manage shared state. * Be mindful of race conditions and deadlocks. * Use context to manage the lifecycle of goroutines. * **Don't Do This:** * Share memory by communicating. * Ignore potential data races. * Forget to handle panics in goroutines. * **Rationale:** Go's concurrency features are powerful but require careful use. Avoiding data races and managing goroutine lifecycles are crucial for building reliable concurrent applications. """go package main import ( "fmt" "sync" ) func worker(id int, jobs <-chan int, results chan<- int, wg *sync.WaitGroup) { defer wg.Done() for j := range jobs { fmt.Printf("worker %d started job %d\n", id, j) // Simulate some work // time.Sleep(time.Second) fmt.Printf("worker %d finished job %d\n", id, j) results <- j * 2 } } func main() { const numJobs = 5 jobs := make(chan int, numJobs) results := make(chan int, numJobs) var wg sync.WaitGroup // Start three workers, initially blocked because there are no jobs. for w := 1; w <= 3; w++ { wg.Add(1) go worker(w, jobs, results, &wg) } // Send 5 jobs over the jobs channel, then close that channel to // indicate that's all the work we have. for j := 1; j <= numJobs; j++ { jobs <- j } close(jobs) // Wait for the workers to finish, then close the results channel. wg.Wait() close(results) // Another possible way to ensure all workers have completed would be: // go func() { // wg.Wait() // close(results) // }() //Finally we collect all the results of the work. for a := range results { fmt.Println(a) } } """ ### 2.6. Dependencies * **Do This:** * Use Go modules to manage dependencies. Lock in your dependencies by committing the "go.mod" and "go.sum" files * Vendor dependencies when necessary for reproducibility and isolation but it is not best practice. * Keep dependencies up to date. * Avoid unnecessary dependencies. * **Don't Do This:** * Manually manage dependencies. * Use vulnerable or unmaintained dependencies. * **Rationale:** Using Go modules ensures reproducible builds and simplifies dependency management. Keeping dependencies up-to-date helps prevent security vulnerabilities. ### 2.7 Keep small functions * **Do This:** Create functions that do one thing well and keep them concise. Aim for functions that are no more than 30-50 lines to improve code clarity. * **Don't Do This:** Create lengthy functions with many responsibilities. * **Rationale** Smaller functions increase readability and testability which in turn enhances code maintainability. ## 3. Ollama-Specific Conventions ### 3.1. Modelfiles * **Do This:** * Follow the [Modelfile documentation](https://github.com/ollama/ollama/blob/main/README.md) for correct syntax and usage. * Use descriptive comments to explain the purpose of each instruction. * Use consistent formatting for instructions and arguments. * Organize instructions logically. * Ensure commands within the Modelfile are idempotent wherever possible. * **Don't Do This:** * Use deprecated or unsupported instructions. * Create Modelfiles that are difficult to understand or maintain. * Hardcode sensitive information in Modelfiles. * **Rationale:** Modelfiles define how models are created and configured in Ollama. Following these conventions ensures that models can be built consistently and reliably. """dockerfile # Modelfile for CodeLlama FROM codellama:7b # Set the temperature for text generation PARAMETER temperature 0.7 # Add a system message SYSTEM """You are a helpful code assistant.""" # Install any necessary dependencies RUN pip install --upgrade pip RUN pip install pandas numpy # Define instruction to guide the model INSTRUCTION """Write me a function that outputs the fibonacci sequence""" """ ### 3.2. Model Management * **Do This:** * Use the "ollama" CLI for managing models. * Organize models into logical namespaces. * Use descriptive tags for models. * **Don't Do This:** * Manually manipulate model files. * Use generic or ambiguous model names. * Neglect to version control your models or configurations. * **Rationale:** Proper model management is essential for reproducibility and collaboration. Consistent naming and tagging make it easier to find and use the right models. ### 3.3. API Usage * **Do This:** * Use the Ollama API to interact with models programmatically. * Handle API errors gracefully. * Use appropriate timeouts and retries. * Sanitize input data before sending it to the API. * **Don't Do This:** * Bypass the API and directly access model files. * Ignore API errors. * Send untrusted data to the API without proper validation. * **Rationale:** The Ollama API provides a consistent and secure way to interact with models. Following these guidelines ensures that your applications are robust and secure. ### 3.4. Logging * **Do This:** * Use structured logging with consistent log levels (e.g., "debug", "info", "warn", "error"). * Include relevant context in log messages (e.g., request ID, user ID). * Log errors with stack traces. * Use a logging library that supports configurable output formats (e.g., JSON). * **Don't Do This:** * Use "fmt.Println" for logging. * Log sensitive information. * Log excessively or insufficiently. * **Rationale:** Structured logging makes it easier to analyze logs and diagnose problems. Consistent log levels and context help prioritize and filter log messages. """go import ( "log" ) func main() { log.Println("This is standard log") log.Printf("This log contains a string %s and an integer %d", "test", 10) log.Fatal("This is a fatal error") log.Panic("This is a panic") } """ ### 3.5 Data Security * **Do This:** * Sanitize all input data to prevent injection attacks. * Store sensitive data securely (e.g., using encryption or key vaults). * Follow security best practices for your programming language and platform. * Regularly review and update security measures. * **Don't Do This:** * Hardcode sensitive information in code or configuration files. * Trust user input without validation. * Ignore security warnings or vulnerabilities. * **Rationale:** Data security is critical to protect user data and prevent unauthorized access. Following security best practices helps minimize the risk of security breaches. ## 4. Testing ### 4.1. Unit Tests * **Do This:** Write unit tests for all functions and methods. Aim for high code coverage. Use table-driven tests to cover various scenarios. Mock external dependencies to isolate units of code. Run unit tests frequently (e.g. via CI/CD pipeline). * **Don't Do This:** Skip unit tests, especially for critical functionality. Write brittle tests that break easily with minor code changes. * **Rationale:** Unit Tests enable early bug detection, allow faster development cycles, promote better code design and increase code reliability in the long run. """go package main import "testing" func add(a, b int) int { return a + b } func TestAdd(t *testing.T) { testCases := []struct { a, b, expected int }{ {1, 2, 3}, {0, 0, 0}, {-1, 1, 0}, {-1, -1, -2}, } for _, tc := range testCases { result := add(tc.a, tc.b) if result != tc.expected { t.Errorf("add(%d, %d) = %d; expected %d", tc.a, tc.b, result, tc.expected) } } } """ ### 4.2. Integration Tests * **Do This:** Write integration tests to ensure that different components of the system work together correctly. Test interactions with external APIs and databases. * **Don't Do This:** Neglect integration tests, leading to undetected integration issues. Rely solely on unit tests, which may not catch problems that arise from interactions between components. * **Rationale:** Integration tests increase confidence that different parts of the system work correctly by testing them to ensure cohesion. ### 4.3. End-to-End Tests * **Do This:** Write end-to-end (E2E) tests to validate the entire system from start to finish. Simulate user interactions to ensure the system behaves as expected. * **Don't Do This:** Skip E2E tests, risking major issues in user-facing features. Make E2E tests too complex or brittle. * **Rationale:** End-to-end tests emulate user behaviour to assert functionality matches business requirements for the user. ### 4.4. Test Data * **Do This:** Generate or carefully select test data to cover all relevant edge cases. Keep test data separate from code. Store test data in a sensible and maintainable format. * **Don't Do This:** Use insufficient test data or ignore edge cases. Mix test data with code. * **Rationale:** Complete test data reduces errors and increases confidence in software quality. ## 5. Performance Optimization ### 5.1. Algorithmic Efficiency * **Do This:**: Choose appropriate algorithms and data structures for the task at hand. Analyze existing code for performance bottlenecks. Opt for algorithms with better time complexity when performance is critical. * **Don't Do This:** Use inefficient or brute-force algorithms without considering their performance implications. Neglect to profile code to identify performance bottlenecks. * **Rationale:** Efficient algorithms ensure quick execution and optimized resource use which is especially important in high-demand environments. ### 5.2. Memory Management * **Do This:** Minimize unnecessary memory allocation and deallocation. Reuse objects when appropriate. Use data structures, such as sync.Pool for temporary object storage. * **Don't Do This:** Create excessive temporary objects that increase garbage collection overhead. Leak memory by failing to release resources. * **Rationale:** Optimizing memory management reduces overheads, speeds up processes and makes systems more responsive. ### 5.3. Concurrency * **Do This:** Use goroutines to perform concurrent operations. Use sync.WaitGroup to manage the lifecycle of goroutines. Use channels for inter-goroutine communication * **Don't Do This:** Create race conditions or deadlocks by incorrectly managing concurrent processes. Overuse concurrency thus negating it's positive potential. * **Rationale:** Optimal concurrency leads to superior resource utilization, increased throughput and responsiveness in systems. ### 5.4. Code Profiling * **Do This:** Use profiling tools (e.g., go tool pprof) to identify performance bottlenecks. Profile code in realistic scenarios to locate areas for improvement. * **Don't Do This:** Make performance optimizations without profiling the code first. Ignore the results of profiling, leading to uninformed adjustments. * **Rationale:** Profiling helps developers precisely identify performance bottlenecks, enabling targeted and effective improvements. ## 6. Security Considerations ### 6.1. Input Validation * **Do This:** Validate and sanitize all user inputs to prevent injection attacks. Use prepared statements for database queries. Use a well-vetted library for input validation and sanitization. * **Don't Do This:** Trust user inputs without validation. Concatenate strings to build database queries. * **Rationale:** Thorough input validation prevents exploits that can compromise the system. ### 6.2. Authentication and Authorization * **Do This:** Establish robust authentication mechanisms to verify user identities. Implement authorization controls to restrict access based on user roles and permissions. Use established and secure authentication protocols (e.g., OAuth 2.0). * **Don't Do This:** Use weak or custom authentication schemes. Grant excessive permissions to users. Store critical data in ways which make it easy to view without authorisation. * **Rationale:** Ensuring strong authentication and authorization policies protects resources and prevent unauthorized access. ### 6.3. Secure Communication * **Do This:** Use HTTPS to protect data in transit. Support HTTPS for all web services and APIs. Enforce the use of TLS 1.3 or higher versions. * **Don't Do This:** use HTTP for sensitive communications. Permit outdated versions of TLS (e.g., TLS 1.0, TLS 1.1). * **Rationale:** Secure communication protocols prevents eavesdropping and tampering, safeguarding data while it is in transit. This document embodies the coding style and conventions that are necessary for contributing to the Ollama project.

# Component Design Standards for Ollama This document outlines the component design standards for the Ollama project. It serves as a guide for developers to create reusable, maintainable, and performant components within the Ollama ecosystem. These standards promote consistency and improve the overall quality of the codebase. These standards should be used when using AI code assistants like GitHub Copilot or Cursor. ## 1. Architectural Principles ### 1.1 Modularity and Loose Coupling **Standard:** Components should be designed as independent modules with well-defined interfaces. Dependencies between components should be minimized. **Why:** Modularity allows for easier testing, modification, and reuse of components. Loose coupling reduces the impact of changes in one component on other parts of the system. **Do This:** * Define clear and concise interfaces for each component. * Use dependency injection to manage dependencies between components. * Favor composition over inheritance to achieve code reuse. **Don't Do This:** * Create tightly coupled components that are difficult to isolate and test. * Introduce circular dependencies between components. * Hardcode dependencies within components. **Example (Go):** """go // Correct: Using interfaces for loose coupling package model // ModelInterface defines the methods a model component should implement. type ModelInterface interface { LoadModel(path string) error Predict(input string) (string, error) UnloadModel() error } // OllamaModel implements ModelInterface type OllamaModel struct { modelPath string // Model specific implementation details ... } func (m *OllamaModel) LoadModel(path string) error { // Implements loading the model from the specified path m.modelPath = path // Implementation... return nil } func (m *OllamaModel) Predict(input string) (string, error) { // Implements processing and getting the model prediction // Implementation... return "predicted output", nil } func (m *OllamaModel) UnloadModel() error { // Implements model unloading and memory freeing // Implementation... return nil } // Incorrect: Tight coupling (avoid this) package model type OllamaModel struct { // Directly dependent on the "logger" package. This makes testing difficult. logger *logger.Logger // Assuming logger is another internal package modelPath string } func (m *OllamaModel) LoadModel(path string, logger *logger.Logger) error { m.modelPath = path m.logger = logger m.logger.Log("Loading model from path: " + path) // Implementation... return nil } """ ### 1.2 Separation of Concerns **Standard:** Each component should have a single, well-defined responsibility. **Why:** Separation of concerns improves code readability, maintainability, and testability. It allows developers to focus on specific aspects of the system. **Do This:** * Divide the system into components based on their functional roles (e.g., data access, business logic, presentation). * Ensure that each component performs only the tasks related to its responsibility. * Minimize the overlap in functionality between components. **Don't Do This:** * Create "god classes" or components that perform too many unrelated tasks. * Mix different concerns within the same component (e.g., mixing data validation with business logic). **Example (Go):** """go // Correct: Separate components for handling requests and model inference package service import ( "fmt" "net/http" "ollama.ai/model" // Assuming ollama.ai/model package ) // InferenceHandler handles HTTP requests for model inference. type InferenceHandler struct { Model model.ModelInterface // Use the interface here for loose coupling } // NewInferenceHandler creates new InferenceHandler instance func NewInferenceHandler(model model.ModelInterface) *InferenceHandler { return &InferenceHandler{Model: model} } func (h *InferenceHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) { // Request parsing, validation, and pre-processing input := r.URL.Query().Get("input") if input == "" { http.Error(w, "Input parameter is required", http.StatusBadRequest) return } // Call to the Model component for inference output, err := h.Model.Predict(input) if err != nil { http.Error(w, fmt.Sprintf("Model prediction error: %v", err), http.StatusInternalServerError) return } // Response formatting and sending w.WriteHeader(http.StatusOK) w.Write([]byte(output)) } // Incorrect: Mixing Concerns package service import ( "fmt" "net/http" "ollama.ai/model" ) // InferenceHandler (Bad Design) handles requests AND model inference. type InferenceHandler struct { Model model.OllamaModel // Directly using the concrete type making testing harder } func (h *InferenceHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) { // Request parsing, validation, pre-processing, and model inference ALL IN ONE PLACE. input := r.URL.Query().Get("input") if input == "" { http.Error(w, "Input parameter is required", http.StatusBadRequest) return } output, err := h.Model.Predict(input) // Directly calling Model's implementation if err != nil { http.Error(w, fmt.Sprintf("Model prediction error: %v", err), http.StatusInternalServerError) return } w.WriteHeader(http.StatusOK) w.Write([]byte(output)) } """ ### 1.3 Single Source of Truth (SSOT) **Standard:** Each piece of data or logic should have a single, authoritative source within the system. **Why:** SSOT prevents inconsistencies and reduces the risk of errors when data or logic needs to be updated. **Do This:** * Identify the authoritative source for each piece of data. * Ensure that all components access data from the single source of truth. * Avoid duplicating data or logic across multiple components. **Don't Do This:** * Store the same data in multiple places without proper synchronization. * Implement the same logic in multiple components. * Allow components to modify data without going through the authoritative source. **Example (Conceptual):** In Ollama, model definitions should have a single, authoritative source, which could be a dedicated configuration file or database. Components that need information about a model (e.g., its architecture, dependencies) should query this source rather than maintaining their own copies of the data. ### 1.4 Abstraction **Standard:** Hide complex implementation details behind simple, well-defined interfaces. **Why:** Abstraction simplifies the usage of components and reduces the impact of internal changes. **Do This:** * Create abstract interfaces that hide implementation details. * Use factory patterns to create instances of components without exposing concrete classes. * Provide clear and concise documentation for the public interfaces of components. **Don't Do This:** * Expose internal implementation details through public interfaces. * Create leaky abstractions that require users to be aware of the underlying implementation. * Over-abstract components, making them overly complex and difficult to use. ## 2. Component Implementation ### 2.1 Naming Conventions **Standard:** Use consistent and descriptive naming conventions for components, classes, methods, and variables. **Why:** Consistent naming improves code readability and maintainability. **Do This:** * Use CamelCase for class names (e.g., "OllamaModel", "RequestHandler"). * Use camelCase for method and variable names (e.g., "loadModel()", "modelPath"). * Use descriptive names that clearly indicate the purpose of the component, class, method, or variable. * Follow the Go naming conventions (e.g., use short, concise names; avoid abbreviations unless they are widely understood). **Don't Do This:** * Use cryptic or ambiguous names. * Use inconsistent naming conventions. * Use names that are too long or verbose. ### 2.2 Error Handling **Standard:** Handle errors gracefully and provide informative error messages. **Why:** Proper error handling prevents crashes and makes it easier to diagnose and resolve problems. **Do This:** * Return errors from functions and methods that can fail. * Check for errors and handle them appropriately (e.g., log the error, return a default value, or propagate the error to the caller). * Provide informative error messages that include context about the error. * Use the "errors" package in Go for creating and handling errors. * Consider using custom error types to provide more specific information about the error. **Don't Do This:** * Ignore errors. * Panic or exit the program without handling the error. * Provide vague or uninformative error messages. **Example (Go):** """go package model import ( "fmt" "errors" ) // Custom error type for model loading failures var ErrModelLoadingFailed = errors.New("failed to load model") type OllamaModel struct { modelPath string } func (m *OllamaModel) LoadModel(path string) error { if path == "" { return fmt.Errorf("model path cannot be empty: %w", ErrModelLoadingFailed) // Wrapping standard error } // Attempt to load the model. err := m.loadFromFile(path) if err != nil { return fmt.Errorf("error loading model from file: %w", err) // Wrapping the underlying error } m.modelPath = path return nil } func (m *OllamaModel) loadFromFile(path string) error { //Simulate loading the model and failing return errors.New("could not read model file") } //Usage example func main() { model := OllamaModel{} err := model.LoadModel("") if err != nil { fmt.Println(err) // prints: model path cannot be empty: failed to load model fmt.Println(errors.Is(err, ErrModelLoadingFailed)) // prints true - showing error wrapping } } """ ### 2.3 Logging **Standard:** Log important events and errors to aid in debugging and monitoring. **Why:** Logging provides valuable insights into the behavior of the system and helps to identify and resolve issues. **Do This:** * Use a logging library (e.g., "logrus", "zap") to log messages. * Log messages at the appropriate level (e.g., debug, info, warn, error). * Include context in log messages (e.g., component name, request ID). * Log errors and warnings along with stack traces. * Consider using structured logging to make it easier to analyze log data. **Don't Do This:** * Log too much or too little information. * Log sensitive information (e.g., passwords, API keys). * Use "fmt.Println" for logging (use a dedicated logging library instead). **Example (Go):** """go package main import ( "time" "github.com/sirupsen/logrus" ) var log = logrus.New() func main() { // You could set this to any io.Writer, see below. log.Out = os.Stdout // You can change the level of logging log.SetLevel(logrus.InfoLevel) log.WithFields(logrus.Fields{ "animal": "walrus", "size": 10, }).Info("A group of walrus emerges from the ocean") log.WithFields(logrus.Fields{ "omg": true, "number": 122, }).Warn("The group's number increased tremendously!") log.WithFields(logrus.Fields{ "omg": true, "number": 100, "id": "ollama-001", //example }).Error("Failed to process request") // Log an error with context time.Sleep(2 * time.Second) } """ ### 2.4 Concurrency **Standard:** Handle concurrency safely and efficiently. **Why:** Ollama often deals with concurrent requests and operations. Proper concurrency management is crucial for performance and stability. **Do This:** * Use goroutines and channels for concurrent execution. * Use mutexes or other synchronization primitives to protect shared resources. * Avoid race conditions and deadlocks. * Use the "sync" package in Go for managing concurrency. * Consider using worker pools to limit the number of concurrent goroutines. **Don't Do This:** * Share mutable state between goroutines without proper synchronization. * Create too many goroutines, which can lead to performance problems. * Block indefinitely in goroutines, which can lead to deadlocks. ### 2.5 Resource Management **Standard:** Manage resources (e.g., memory, file handles, network connections) carefully to prevent leaks. **Why:** Resource leaks can lead to performance degradation and instability. **Do This:** * Use "defer" to ensure that resources are released when they are no longer needed. * Close file handles and network connections explicitly. * Free memory that is no longer being used. * Use garbage collection efficiently (e.g., avoid creating unnecessary objects). **Don't Do This:** * Forget to release resources. * Allocate large amounts of memory without a clear need. * Create circular references that prevent garbage collection. **Example (Go):** """go package main import ( "fmt" "os" ) func main() { // Open a file. file, err := os.Open("my_file.txt") if err != nil { fmt.Println("Error opening file:", err) return } defer file.Close() //Ensure file is closed when function exits. // Read from the file. buffer := make([]byte, 100) _, err = file.Read(buffer) if err != nil { fmt.Println("Error reading file:", err) return } fmt.Println("Read:", string(buffer)) } """ ### 2.6 Testing **Standard:** Write unit tests, integration tests, and end-to-end tests to ensure code quality and prevent regressions. **Why:** Testing is crucial for verifying the correctness of the code and preventing bugs from being introduced. **Do This:** * Write unit tests for individual components and functions. * Write integration tests to verify that components work together correctly. * Write end-to-end tests to verify that the entire system works as expected. * Use a testing framework (e.g., "testing" package in Go). * Automate the execution of tests. * Aim for high test coverage. **Don't Do This:** * Skip writing tests. * Write tests that are too brittle (i.e., that break easily when the code is changed). * Write tests that are too slow to execute. ## 3. Ollama-Specific Considerations ### 3.1 Modelfiles When working with modelfiles, treat them as configuration as code. Changes should be automated via tooling, not manual edits. ### 3.2 Interfacing with External Libraries Ollama often integrates with external libraries for tasks like model loading, inference, and data processing. It's important to handle these integrations carefully. **Standard:** * Use clear and well-defined interfaces to abstract the interaction with external libraries. * Implement error handling to gracefully handle failures in external libraries. * Consider using dependency injection to make it easier to swap out different implementations of external libraries. **Example:** If using a C++ library for model inference, create a Go package that wraps the C++ library and provides a Go-friendly interface. Handle any errors that occur in the C++ library and translate them into Go errors. ### 3.3 Performance Optimization Ollama is often used for computationally intensive tasks. Performance is a critical consideration. **Standard:** * Profile the code to identify performance bottlenecks. * Use efficient algorithms and data structures. * Optimize memory usage. * Use concurrency to take advantage of multi-core processors. * Consider using caching to reduce the load on external resources. **Example:** When performing model inference, consider using batching to process multiple inputs at once. This can improve throughput by amortizing the overhead of model loading and inference. Use tools like "pprof" to diagnose performance issues. ### 3.4 Security Ollama may handle sensitive data. Security is a paramount concern. **Standard:** * Follow security best practices for all code. * Validate all inputs to prevent injection attacks. * Protect sensitive data (e.g., API keys, passwords) using encryption. * Regularly audit the code for security vulnerabilities. * Be aware of common LLM security vulnerabilities such as prompt injection, and implement safeguards. **Example:** When handling user input, sanitize the input to prevent cross-site scripting (XSS) and other injection attacks. Use a secure storage mechanism to store API keys and passwords. ## 4. Modern Approaches and Patterns Ollama should strive to incorporate modern software development principles and well-established design patterns. ### 4.1 Dependency Injection As shown in prior examples, use dependency injection to decouple components. ### 4.2 Functional Programming Where appropriate leverage functional programming concepts such as pure functions, immutability, and higher-order functions to improve code clarity and testability. ### 4.3 Observability Implement robust monitoring and tracing to gain insights into the performance and behavior of Ollama components in production. ### 4.4 Asynchronous Programming Employ asynchronous programming techniques such as goroutines and channels to handle concurrent requests efficiently and prevent blocking operations. ### 4.5 Immutability Ensure that once an object is created, its state cannot be changed. This helps to avoid race conditions and makes it easier to reason about the code. ## 5. Anti-Patterns and Common Mistakes ### 5.1 Premature Optimization Do not optimize code before profiling and identifying actual performance bottlenecks. ### 5.2 Reinventing the Wheel Leverage existing libraries and frameworks instead of implementing functionality from scratch. ### 5.3 Ignoring Errors As previously mentioned, ignoring errors can lead to unpredictable behavior and make it difficult to debug issues. ### 5.4 Over-Engineering Avoid creating overly complex solutions when simpler alternatives exist. This can increase the maintenance burden and make the code harder to understand. ### 5.5 God Classes Avoid creating classes that have too many responsibilities, as this can make the code harder to maintain and test. By following these component design standards, Ollama developers can create a codebase that is robust, maintainable, and performant. This will lead to a better user experience and a more successful project.

# State Management Standards for Ollama This document outlines coding standards for managing state within Ollama, promoting maintainability, performance, and security. It focuses on application state, data flow, and reactivity, within the context of Ollama. ## 1. Introduction Effective state management is crucial for building robust and scalable Ollama applications. Choosing the right approach impacts application performance, data consistency, and the overall developer experience. These standards guide developers in selecting and implementing appropriate state management techniques for various scenarios within Ollama. ## 2. Core Principles * **Explicit State Definition:** Clearly define the application state and its scope. Avoid implicit or hidden state that can lead to unexpected behavior. * **Single Source of Truth:** Designate a single source of truth for each piece of state. This minimizes inconsistencies and simplifies debugging. * **Immutability:** Prefer immutable state updates. Immutable data structures help prevent accidental modifications and simplify reasoning about state changes. * **Unidirectional Data Flow:** Establish a clear and predictable flow of data. Changes should originate from a central point and propagate through the application in a defined manner. * **Reactive Updates:** Embrace reactivity to automatically update dependent components when state changes. * **Minimize Global State:** Avoid excessive use of global state, favoring localized state management for improved modularity and testability. * **Optimize for Performance:** Evaluate the performance impact of state management choices, especially in resource-constrained environments. ## 3. State Management Approaches ### 3.1. Local Component State * **Description:** Managing state within individual components. Suitable for simple UI elements or data that doesn't need to be shared across the application. Within Ollama, this might involve managing the temporary state of a UI element used for model selection or a configuration form. * **Standards:** * **Do This:** Use local component state ("useState", "useReducer" patterns as appropriate for the specific UI framework, if applicable - Ollama itself has minimal UI) for managing component-specific data that doesn't need to be shared. * **Don't Do This:** Overuse local state for data that should be shared or persisted. This leads to duplication and inconsistencies. * **Why:** Simplifies component logic and reduces dependencies. * **Example (Conceptual, assuming Ollama exposes a UI via a framework like React):** """javascript // Hypothetical React component for selecting a model import React, { useState } from 'react'; function ModelSelector() { const [selectedModel, setSelectedModel] = useState(''); const handleModelChange = (event) => { setSelectedModel(event.target.value); }; return ( <div> <label htmlFor="modelSelect">Select a Model:</label> <select id="modelSelect" value={selectedModel} onChange={handleModelChange}> <option value="llama2">Llama 2</option> <option value="mistral">Mistral</option> {/* more models */} </select> <p>You selected: {selectedModel}</p> </div> ); } export default ModelSelector; """ ### 3.2. Global State Management (Minimal in Ollama Core) * **Description:** Managing state that is accessible and modifiable from anywhere in the application. Less applicable to Ollama's core, but relevant for UI clients built around it. Consider configurations, user preferences, or authentication status. * **Standards:** * **Do This:** Use global state management sparingly. Only for data that is truly shared across the entire application. Consider solutions like React Context (if a React-based UI is present) or simple configuration files if a UI framework is not the primary client interaction. * **Don't Do This:** Use global state as a dumping ground for all application data. This leads to tight coupling and makes debugging difficult. * **Do This:** When using configuration files, prefer formats like YAML or JSON for readability and ease of parsing. * **Do This:** Implement clear mechanisms for updating the global state, such as dedicated functions or classes. * **Why:** Provides a centralized location for managing shared data, but increases complexity. * **Example (Conceptual, Ollama-specific configuration):** """yaml # Ollama Configuration (ollama.yaml) model_path: /opt/ollama/models default_model: llama2 temperature: 0.75 num_gpu: 1 """ """python # Python snippet to load and access configuration (demonstrates principle) import yaml def load_config(filepath="ollama.yaml"): with open(filepath, 'r') as f: config = yaml.safe_load(f) return config config = load_config() default_model = config.get("default_model") print(f"Default model: {default_model}") """ In a more complex UI-driven scenario, you might use React Context to manage settings read from a config file: """javascript // SettingsContext.js (Conceptual - this might not exist directly in Ollama core) import React, { createContext, useState, useEffect } from 'react'; export const SettingsContext = createContext(); export const SettingsProvider = ({ children }) => { const [settings, setSettings] = useState({}); useEffect(() => { // Simulate fetching settings from a YAML file or API const fetchSettings = async () => { // In a real application, use a more robust method to load settings const loadedSettings = { defaultModel: 'llama2', temperature: 0.75, }; setSettings(loadedSettings); }; fetchSettings(); }, []); return ( <SettingsContext.Provider value={{ settings, setSettings }}> {children} </SettingsContext.Provider> ); }; """ ### 3.3. Data Stores and Databases * **Description:** Persisting application state to disk or a database. This is critical for Ollama to store models, metadata, and potentially user-specific preferences. * **Standards:** * **Do This:** Choose a data store that aligns with the application's requirements. Consider key-value stores (e.g., LevelDB) for simple data, SQLite for relational data that doesn't require a full-blown database server, or potentially a full database like Postgres if complex relationships and queries are needed. * **Do This:** Use an ORM (Object-Relational Mapper) or ODM (Object-Document Mapper) to simplify database interactions and prevent SQL injection vulnerabilities *if and only if* a full-blown relational database is chosen. This is probably overkill for most of Ollama's internal state. * **Don't Do This:** Directly embed SQL queries in your code. This makes the code harder to maintain and opens the door to security vulnerabilities. * **Do This:** Implement proper data validation and sanitization to prevent data corruption and security issues. * **Do This:** Use asynchronous operations for database interactions to avoid blocking the main thread. * **Do This:** Apply migrations for schema changes and use version control for tracking these changes. * **Consider This:** Ollama already appears to use a specific file structure for storing models. Extending this existing structure may be preferable to introducing a completely new database if possible. * **Why:** Provides persistent storage for application data. * **Example (Conceptual - using SQLite for storing model metadata):** """python # Python example using SQLite (demonstrates principles) import sqlite3 def create_connection(db_file): """Create a database connection to a SQLite database.""" conn = None try: conn = sqlite3.connect(db_file) return conn except sqlite3.Error as e: print(e) return conn def create_table(conn, create_table_sql): """Create a table from the create_table_sql statement.""" try: c = conn.cursor() c.execute(create_table_sql) except sqlite3.Error as e: print(e) def create_model(conn, model): """Create a new model record.""" sql = ''' INSERT INTO models(name, path, size, created_at) VALUES(?,?,?,?) ''' cur = conn.cursor() cur.execute(sql, model) conn.commit() return cur.lastrowid def main(): database = "ollama_models.db" sql_create_models_table = """ CREATE TABLE IF NOT EXISTS models ( id integer PRIMARY KEY, name text NOT NULL, path text, size integer, created_at text ); """ conn = create_connection(database) if conn is not None: create_table(conn, sql_create_models_table) # Example: Model creation model = ('Llama 2', '/opt/ollama/models/llama2', 4000000000, '2024-01-01') # example size in bytes model_id = create_model(conn, model) print(f"Created model with id: {model_id}") else: print("Error! cannot create the database connection.") if conn: conn.close() if __name__ == '__main__': main() """ ### 3.4. State Machines * **Description:** Modeling application state as a finite state machine. Useful for managing complex, state-dependent logic. This may be applicable to managing the state of an Ollama model as it's being loaded, processed, or unloaded. * **Standards:** * **Do This:** Define clear states and transitions between states. Use a library or framework to manage the state machine logic. * **Don't Do This:** Implement state machine logic manually with nested "if" statements. This makes the code harder to understand and maintain. * **Do This:** Ensure that all states are well-defined and that transitions are handled consistently. * **Do This:** Properly handle edge cases and transitions to error states. * **Why:** Provides a structured way to manage complex state transitions. * **Example (Conceptual - using a Python library like "transitions"):** """python from transitions import Machine class ModelSession(object): def __init__(self): self.state = "idle" # Initial state def load(self): print("Loading the model...") def ready(self): print("Model is ready to generate...") def generate(self): print("Generating output...") def unload(self): print("Unloading the model...") def error(self): print("An error occurred.") session = ModelSession() machine = Machine(model=session, states=['idle', 'loading', 'ready', 'generating', 'unloading', 'error'], initial='idle', transitions=[ {'trigger': 'start_load', 'source': 'idle', 'dest': 'loading', 'before': 'load'}, {'trigger': 'load_complete', 'source': 'loading', 'dest': 'ready', 'after': 'ready'}, {'trigger': 'start_generate', 'source': 'ready', 'dest': 'generating', 'before': 'generate'}, {'trigger': 'generation_complete', 'source': 'generating', 'dest': 'ready'}, {'trigger': 'unload_model', 'source': 'ready', 'dest': 'unloading', 'before': 'unload'}, {'trigger': 'complete_unload', 'source': 'unloading', 'dest': 'idle'}, {'trigger': 'report_error', 'source': '*', 'dest': 'error', 'before': 'error'} ], auto_transitions=False, # Disable automatic transitions ) # Example usage print(f"Current state: {session.state}") session.start_load() print(f"Current state: {session.state}") session.load_complete() print(f"Current state: {session.state}") session.start_generate() print(f"Current state: {session.state}") session.generation_complete() print(f"Current state: {session.state}") session.unload_model() print(f"Current state: {session.state}") session.complete_unload() print(f"Current state: {session.state}") """ ### 3.5. Reactivity (If UI is present) * **Description:** Automatically updating dependent components or data when state changes. If Ollama exposes a UI (e.g., through a separate web server), reactivity can be handled using libraries like RxJS, or MobX. * **Standards:** * **Do This:** Use reactive programming techniques to simplify data flow and automatically update components in response to state changes. * **Don't Do This:** Manually update components or listeners when state changes. This leads to brittle code and missed updates. * **Do This:** Use appropriate operators to transform and filter data streams to prevent unnecessary updates and improve performance. * **Why:** Simplifies data flow and improves UI responsiveness. * **Example (Conceptual, using RxJS):** """javascript // Conceptual Example (Illustrative - Ollama core might not use this directly) import { fromEvent, map, debounceTime } from 'rxjs'; // Assuming an input field for configuring model temperature const temperatureInput = document.getElementById('temperatureInput'); // Create an observable from the input field's 'input' event const temperature$ = fromEvent(temperatureInput, 'input') .pipe( map(event => parseFloat(event.target.value)), // Extract temperature value debounceTime(300) // Debounce to prevent excessive updates ); // Subscribe to the temperature observable and update the Ollama config temperature$.subscribe(temperature => { // In a real application, this would update the Ollama configuration console.log("Updating Ollama temperature to: ${temperature}"); // Example: Call a function to update the Ollama config (placeholder) // updateOllamaTemperature(temperature); }); """ ## 4. Data Flow Patterns ### 4.1. Unidirectional Data Flow * **Description:** Establishing a clear and predictable data flow. Changes originate from a central point and propagate through the application in a defined manner. * **Standards:** * **Do This:** Use a unidirectional data flow architecture. This is commonly associated with UI frameworks, but the principle applies – avoid cycles in your data dependencies. * **Don't Do This:** Mutate state directly from different parts of the application. This leads to unpredictable behavior. * **Do This:** Define clear actions or events that trigger state changes. * **Why:** Improves predictability and makes debugging easier. ### 4.2. Event Sourcing (Advanced) * **Description:** Storing all state changes as a sequence of events. This allows for time-travel debugging, auditing, and replaying events to reconstruct the application state. May be overkill for most Ollama use cases. * **Standards:** * **Do This:** Append all state changes to an event log. * **Don't Do This:** Directly mutate the application state. Instead, apply events to the state. * **Do This:** Use an append-only data store for the event log to ensure immutability. * **Why:** Provides a detailed history of state changes and enables advanced features. ## 5. Common Anti-Patterns * **Global Variable Abuse:** Using global variables to store application state leads to tight coupling and makes debugging difficult. Avoid this. * **Direct State Mutation:** Directly modifying state objects can lead to unexpected behavior and make it difficult to track changes. Prefer immutable updates. * **Ignoring Performance:** Neglecting the performance impact of state management choices can lead to slow UI updates and a poor user experience. Optimize for performance, especially when dealing with large datasets. * **Lack of Data Validation:** Failing to validate data before storing it can lead to data corruption and security vulnerabilities. Implement proper data validation. * **Over-Complicating Things:** Choosing overly complex state management solutions for simple applications can add unnecessary overhead. Start with simpler solutions and only introduce complexity when needed. * **Magic Strings:** Avoid hardcoding strings associated with states and actions. Use constants instead. ## 6. Security Considerations * **Data Sanitization:** Sanitize all user inputs before storing them in the application state or database. This prevents cross-site scripting (XSS) and SQL injection vulnerabilities. * **Access Control:** Implement proper access controls to protect sensitive data in the application state. * **Secure Storage:** Store sensitive data (e.g., API keys, passwords) securely using encryption or other appropriate techniques. * **Regular Updates:** Keep all state management libraries and dependencies up to date to patch security vulnerabilities. ## 7. Performance Optimization * **Memoization:** Use memoization techniques to avoid re-rendering components or recomputing values unnecessarily. * **Lazy Loading:** Load data on demand to reduce the initial load time and improve performance. * **Virtualization:** Use virtualization techniques to efficiently render large lists or tables of data. * **Debouncing and Throttling:** Use debouncing and throttling to limit the frequency of updates and improve performance. ## 8. Conclusion These coding standards provide a foundation for effective state management within Ollama. By following these guidelines, developers can create more maintainable, performant, and secure applications. It needs to be applied judiciously, given the highly efficient and streamlined nature of Ollama. Prioritize simplicity and performance while adhering to the principles outlined above. Remember that the "best" approach always depends on the specific requirements of the application.