Ollama MCP Proxy

A comprehensive Model Context Protocol (MCP) proxy server that bridges MCP clients with Ollama's local language models, providing advanced features like RAG integration, context management, caching, and production-ready security.

🌟 Features

Core Functionality

• MCP Protocol Implementation: Full server-side MCP support with tools, resources, and prompts

• Ollama Integration: Seamless connection to local Ollama language models

• Multiple Transport Methods: HTTP with Server-Sent Events (SSE) and WebSocket support

• Advanced Context Management: Session-based isolation with conversation branching and merging

Advanced AI Capabilities

• RAG Integration: Vector-based document retrieval with FAISS and sentence transformers

• Knowledge Base Connectivity: Integration with external knowledge sources

• Advanced Summarization: Context window management with intelligent summarization

• Multi-Model Support: Dynamic model discovery and switching

Performance & Production Features

• Intelligent Caching: Multi-tier caching with Redis and local fallback

• Circuit Breaker Pattern: Fault tolerance with automatic recovery

• Rate Limiting: Configurable request throttling and protection

• Streaming Optimization: Efficient real-time response streaming

Security & Authentication

• OAuth 2.0 Support: Comprehensive authentication and authorization

• Role-Based Access Control (RBAC): Granular permission management

• Data Encryption: At-rest encryption for sensitive conversation data

• Security Headers: Production-ready security configuration

Developer Experience

• Comprehensive Testing: Unit, integration, and load testing suites

• Development Tools: Hot reload, profiling, and debugging support

• Structured Logging: JSON-formatted logs with correlation IDs

• Configuration Management: Environment-based configuration with validation

Related MCP server: FastMCP

🏗️ Architecture

┌─────────────────┐    ┌──────────────────────┐    ┌─────────────────┐
│   MCP Client    │    │   Ollama MCP Proxy   │    │  Ollama Server  │
│  (Claude, etc.) │◄──►│                      │◄──►│   (Local AI)    │
└─────────────────┘    └──────────────────────┘    └─────────────────┘
                              │
                              ▼
                       ┌─────────────────┐
                       │  Configuration  │
                       │   & Storage     │
                       └─────────────────┘

Key Components

• OllamaMCPServer: Main MCP server implementation with tool and resource handlers

• OllamaClient: Robust HTTP client with retry logic and circuit breaker

• ContextManager: Sophisticated session management with branching and search

• RAG Integration: Vector-based document retrieval and knowledge augmentation

• Security Framework: Authentication, authorization, and data protection

• Cache System: Multi-level caching with intelligent warming and invalidation

🚀 Quick Start

Prerequisites

• Python 3.8 or higher

• Ollama installed and running locally

• Redis (optional, for distributed caching)

Installation

1. Clone the repository:

git clone https://github.com/ollama-mcp-proxy/ollama-mcp-proxy.git
cd ollama-mcp-proxy

2. Create and activate virtual environment:

python -m venv venv
# Windows
venv\Scripts\activate
# Unix/macOS
source venv/bin/activate

3. Install dependencies:

pip install -r requirements.txt

4. Start Ollama (if not already running):

ollama serve

5. Run the MCP proxy:

python -m ollama_mcp_proxy.server --config config/development.json

Or using the CLI:

ollama-mcp-proxy --config config/development.json

📝 Configuration

The proxy uses JSON configuration files for different environments:

• config/development.json - Development settings with debug mode

• config/production.json - Production-ready configuration

Key Configuration Sections

{
  "ollama": {
    "host": "localhost",
    "port": 11434,
    "timeout": 30,
    "max_retries": 3
  },
  "mcp": {
    "port": 8000,
    "transport": "http",
    "auth_enabled": false
  },
  "cache": {
    "enabled": true,
    "type": "hybrid",
    "redis": {
      "enabled": true,
      "host": "localhost",
      "port": 6379
    }
  },
  "rag": {
    "enabled": false,
    "vector_store": "faiss",
    "embedding_model": "all-MiniLM-L6-v2"
  }
}

Environment Variables

• OLLAMA_HOST - Ollama server host (default: localhost:11434)

• MCP_PROXY_PORT - MCP proxy port (default: 8000)

• OLLAMA_MCP_CONFIG - Path to configuration file

🔧 Claude Desktop Integration

Add to your Claude Desktop MCP configuration:

{
  "mcpServers": {
    "ollama-proxy": {
      "command": "python",
      "args": ["-m", "ollama_mcp_proxy"],
      "env": {
        "OLLAMA_HOST": "localhost:11434",
        "MCP_PROXY_PORT": "8000"
      }
    }
  }
}

🛠️ Available Tools

The proxy exposes several MCP tools:

Text Completion

{
  "name": "ollama_completion",
  "arguments": {
    "prompt": "Explain quantum computing",
    "model": "llama2",
    "temperature": 0.7,
    "max_tokens": 500
  }
}

Code Completion

{
  "name": "code_completion",
  "arguments": {
    "code": "def factorial(n):",
    "language": "python",
    "model": "codellama"
  }
}

Tool Chaining

{
  "name": "tool_chain",
  "arguments": {
    "tools": [
      {"tool": "research", "args": {"topic": "AI ethics"}},
      {"tool": "summarize", "args": {"input": "{{previous}}"}}
    ]
  }
}

📚 Resources

MCP resources provide access to:

• Model Information: /models/{model_name} - Model capabilities and metadata

• System Status: /system/status - Health and performance metrics

• Configuration: /config/current - Current configuration settings

• Session Info: /sessions/{session_id} - Session context and history

🧪 Development

Setup Development Environment

1. Install development dependencies:

pip install -e ".[dev]"

2. Set up pre-commit hooks:

pre-commit install

3. Run in development mode:

python -m ollama_mcp_proxy.server --config config/development.json --debug

Code Quality Tools

• Black: Code formatting

• isort: Import sorting

• flake8: Linting

• mypy: Type checking

# Format code
black src/ tests/

# Sort imports
isort src/ tests/

# Run linting
flake8 src/ tests/

# Type checking
mypy src/

🧪 Testing

The project includes comprehensive testing with pytest:

Running Tests

# Run all tests
pytest

# Run with coverage
pytest --cov=ollama_mcp_proxy --cov-report=html

# Run specific test categories
pytest -m unit          # Unit tests only
pytest -m integration   # Integration tests only
pytest -m load          # Load tests only

# Run specific test file
pytest tests/test_auth.py -v

Test Categories

• Unit Tests: Individual component testing with mocked dependencies

• Integration Tests: End-to-end testing with real Ollama integration

• Load Tests: Performance and concurrency testing

Test Configuration

Tests use comprehensive fixtures defined in conftest.py:

• Mock Ollama client with predictable responses

• Sample test data and configurations

• Error scenario simulation

• Async testing support

📊 Performance Features

Caching Strategy

• Response Caching: Intelligent caching with TTL-based expiration

• Model Output Caching: Ollama response caching for repeated queries

• Cache Warming: Proactive cache population for popular models

• Distributed Caching: Redis integration for multi-instance deployments

Memory Management

• Context Window Sliding: Automatic context truncation for long conversations

• Memory Pressure Handling: Automatic cleanup when memory limits are reached

• Session Compression: Zlib compression for inactive sessions

• Garbage Collection: Efficient cleanup of expired sessions

Circuit Breaker

• Fault Tolerance: Automatic failure detection and recovery

• Exponential Backoff: Intelligent retry strategies

• Health Monitoring: Continuous health checking of dependencies

🔒 Security Features

Authentication & Authorization

• API Key Authentication: Secure key-based access control

• OAuth 2.0 Integration: Industry-standard authentication

• Role-Based Access Control: Granular permission management

• JWT Token Support: Stateless authentication with JSON Web Tokens

Data Protection

• Encryption at Rest: AES encryption for stored conversation data

• Request Sanitization: Input validation and sanitization

• Audit Logging: Comprehensive security event logging

• Security Headers: CORS, CSP, and other security headers

Rate Limiting

• Per-User Limits: Individual user rate limiting

• Global Limits: System-wide protection against abuse

• Sliding Window: Advanced rate limiting algorithms

🚀 Production Deployment

Docker Deployment

# Build Docker image
docker build -t ollama-mcp-proxy .

# Run with Docker Compose
docker-compose up -d

Environment Configuration

# Production environment variables
export OLLAMA_MCP_CONFIG=/app/config/production.json
export REDIS_URL=redis://localhost:6379
export LOG_LEVEL=INFO

Monitoring

• Health Endpoints: /health and /metrics endpoints

• Structured Logging: JSON logs with correlation IDs

• Performance Metrics: Request/response time tracking

• Error Rate Monitoring: Comprehensive error tracking

🤝 Contributing

We welcome contributions! Please see our Contributing Guidelines for details.

Development Workflow

1. Fork the repository

2. Create a feature branch

3. Make your changes with tests

4. Run the test suite

5. Submit a pull request

Code Style

• Follow PEP 8 style guidelines

• Use type hints throughout

• Write comprehensive docstrings

• Maintain test coverage above 90%

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Ollama for providing the local language model API

• Model Context Protocol for the protocol specification

• Anthropic for MCP development and Claude integration

• All contributors who help make this project better

📞 Support

• Issues: GitHub Issues

• Discussions: GitHub Discussions

• Documentation: Read the Docs

Built with ❤️ for the MCP community