README.md•14.6 kB
# MCP AI Hub
[](https://www.python.org/downloads/) [](LICENSE) [](https://github.com/psf/black) [](https://github.com/astral-sh/ruff) [](https://pepy.tech/projects/mcp-ai-hub)
A Model Context Protocol (MCP) server that provides unified access to various AI providers through LiteLM. Chat with OpenAI, Anthropic, and 100+ other AI models using a single, consistent interface.
## 🌟 Overview
MCP AI Hub acts as a bridge between MCP clients (like Claude Desktop/Code) and multiple AI providers. It leverages LiteLM's unified API to provide seamless access to 100+ AI models without requiring separate integrations for each provider.
**Key Benefits:**
- **Unified Interface**: Single API for all AI providers
- **100+ Providers**: OpenAI, Anthropic, Google, Azure, AWS Bedrock, and more
- **MCP Protocol**: Native integration with Claude Desktop and Claude Code
- **Flexible Configuration**: YAML-based configuration with Pydantic validation
- **Multiple Transports**: stdio, SSE, and HTTP transport options
- **Custom Endpoints**: Support for proxy servers and local deployments
## Quick Start
### 1. Install
Choose your preferred installation method:
```bash
# Option A: Install from PyPI
pip install mcp-ai-hub
# Option B: Install with uv (recommended)
uv tool install mcp-ai-hub
# Option C: Install from source
pip install git+https://github.com/your-username/mcp-ai-hub.git
```
**Installation Notes:**
- `uv` is a fast Python package installer and resolver
- The package requires Python 3.10 or higher
- All dependencies are automatically resolved and installed
### 2. Configure
Create a configuration file at `~/.ai_hub.yaml` with your API keys and model configurations:
```yaml
model_list:
- model_name: gpt-4 # Friendly name you'll use in MCP tools
litellm_params:
model: openai/gpt-4 # LiteLM provider/model identifier
api_key: "sk-your-openai-api-key-here" # Your actual OpenAI API key
max_tokens: 2048 # Maximum response tokens
temperature: 0.7 # Response creativity (0.0-1.0)
- model_name: claude-sonnet
litellm_params:
model: anthropic/claude-3-5-sonnet-20241022
api_key: "sk-ant-your-anthropic-api-key-here"
max_tokens: 4096
temperature: 0.7
```
**Configuration Guidelines:**
- **API Keys**: Replace placeholder keys with your actual API keys
- **Model Names**: Use descriptive names you'll remember (e.g., `gpt-4`, `claude-sonnet`)
- **LiteLM Models**: Use LiteLM's provider/model format (e.g., `openai/gpt-4`, `anthropic/claude-3-5-sonnet-20241022`)
- **Parameters**: Configure `max_tokens`, `temperature`, and other LiteLM-supported parameters
- **Security**: Keep your config file secure with appropriate file permissions (chmod 600)
### 3. Connect to Claude Desktop
Configure Claude Desktop to use MCP AI Hub by editing your configuration file:
**macOS:** `~/Library/Application Support/Claude/claude_desktop_config.json`
**Windows:** `%APPDATA%\Claude\claude_desktop_config.json`
```json
{
"mcpServers": {
"ai-hub": {
"command": "mcp-ai-hub"
}
}
}
```
### 4. Connect to Claude Code
```sh
claude mcp add -s user ai-hub mcp-ai-hub
```
## Advanced Usage
### CLI Options and Transport Types
MCP AI Hub supports multiple transport mechanisms for different use cases:
**Command Line Options:**
```bash
# Default stdio transport (for MCP clients like Claude Desktop)
mcp-ai-hub
# Server-Sent Events transport (for web applications)
mcp-ai-hub --transport sse --host 0.0.0.0 --port 3001
# Streamable HTTP transport (for direct API calls)
mcp-ai-hub --transport http --port 8080
# Custom config file and debug logging
mcp-ai-hub --config /path/to/config.yaml --log-level DEBUG
```
**Transport Type Details:**
| Transport | Use Case | Default Host:Port | Description |
|-----------|----------|-------------------|-------------|
| `stdio` | MCP clients (Claude Desktop/Code) | N/A | Standard input/output, default for MCP |
| `sse` | Web applications | localhost:3001 | Server-Sent Events for real-time web apps |
| `http` | Direct API calls | localhost:3001 (override with `--port`) | HTTP transport with streaming support |
**CLI Arguments:**
- `--transport {stdio,sse,http}`: Transport protocol (default: stdio)
- `--host HOST`: Host address for SSE/HTTP (default: localhost)
- `--port PORT`: Port number for SSE/HTTP (default: 3001; override if you need a different port)
- `--config CONFIG`: Custom config file path (default: ~/.ai_hub.yaml)
- `--log-level {DEBUG,INFO,WARNING,ERROR}`: Logging verbosity (default: INFO)
## Usage
Once MCP AI Hub is connected to your MCP client, you can interact with AI models using these tools:
### MCP Tool Reference
**Primary Chat Tool:**
```python
chat(model_name: str, message: str | list[dict]) -> str
```
- **model_name**: Name of the configured model (e.g., "gpt-4", "claude-sonnet")
- **message**: String message or OpenAI-style message list
- **Returns**: AI model response as string
**Model Discovery Tools:**
```python
list_models() -> list[str]
```
- **Returns**: List of all configured model names
```python
get_model_info(model_name: str) -> dict
```
- **model_name**: Name of the configured model
- **Returns**: Model configuration details including provider, parameters, etc.
## Configuration
MCP AI Hub supports 100+ AI providers through LiteLM. Configure your models in `~/.ai_hub.yaml` with API keys and custom parameters.
### System Prompts
You can define system prompts at two levels:
- `global_system_prompt`: Applied to all models by default
- Per-model `system_prompt`: Overrides the global prompt for that model
Precedence: model-specific prompt > global prompt. If a model's `system_prompt` is set to an empty string, it disables the global prompt for that model.
```yaml
global_system_prompt: "You are a helpful AI assistant. Be concise."
model_list:
- model_name: gpt-4
system_prompt: "You are a precise coding assistant."
litellm_params:
model: openai/gpt-4
api_key: "sk-your-openai-api-key"
- model_name: claude-sonnet
# Empty string disables the global prompt for this model
system_prompt: ""
litellm_params:
model: anthropic/claude-3-5-sonnet-20241022
api_key: "sk-ant-your-anthropic-api-key"
```
Notes:
- The server prepends the configured system prompt to the message list it sends to providers.
- If you pass an explicit message list that already contains a `system` message, both system messages will be included in order (configured prompt first).
### Supported Providers
**Major AI Providers:**
- **OpenAI**: GPT-4, GPT-3.5-turbo, GPT-4-turbo, etc.
- **Anthropic**: Claude 3.5 Sonnet, Claude 3 Haiku, Claude 3 Opus
- **Google**: Gemini Pro, Gemini Pro Vision, Gemini Ultra
- **Azure OpenAI**: Azure-hosted OpenAI models
- **AWS Bedrock**: Claude, Llama, Jurassic, and more
- **Together AI**: Llama, Mistral, Falcon, and open-source models
- **Hugging Face**: Various open-source models
- **Local Models**: Ollama, LM Studio, and other local deployments
**Configuration Parameters:**
- **api_key**: Your provider API key (required)
- **max_tokens**: Maximum response tokens (optional)
- **temperature**: Response creativity 0.0-1.0 (optional)
- **api_base**: Custom endpoint URL (for proxies/local servers)
- **Additional**: All LiteLM-supported parameters
### Configuration Examples
**Basic Configuration:**
```yaml
global_system_prompt: "You are a helpful AI assistant. Be concise."
model_list:
- model_name: gpt-4
system_prompt: "You are a precise coding assistant." # overrides global
litellm_params:
model: openai/gpt-4
api_key: "sk-your-actual-openai-api-key"
max_tokens: 2048
temperature: 0.7
- model_name: claude-sonnet
litellm_params:
model: anthropic/claude-3-5-sonnet-20241022
api_key: "sk-ant-your-actual-anthropic-api-key"
max_tokens: 4096
temperature: 0.7
```
**Custom Parameters:**
```yaml
model_list:
- model_name: gpt-4-creative
litellm_params:
model: openai/gpt-4
api_key: "sk-your-openai-key"
max_tokens: 4096
temperature: 0.9 # Higher creativity
top_p: 0.95
frequency_penalty: 0.1
presence_penalty: 0.1
- model_name: claude-analytical
litellm_params:
model: anthropic/claude-3-5-sonnet-20241022
api_key: "sk-ant-your-anthropic-key"
max_tokens: 8192
temperature: 0.3 # Lower creativity for analytical tasks
stop_sequences: ["\n\n", "Human:"]
```
**Local LLM Server Configuration:**
```yaml
model_list:
- model_name: local-llama
litellm_params:
model: openai/llama-2-7b-chat
api_key: "dummy-key" # Local servers often accept any API key
api_base: "http://localhost:8080/v1" # Local OpenAI-compatible server
max_tokens: 2048
temperature: 0.7
```
For more providers, please refer to the LiteLLM docs: <https://docs.litellm.ai/docs/providers>.
## Development
**Setup:**
```bash
# Install all dependencies including dev dependencies
uv sync
# Install package in development mode
uv pip install -e ".[dev]"
# Add new runtime dependencies
uv add package_name
# Add new development dependencies
uv add --dev package_name
# Update dependencies
uv sync --upgrade
```
**Running and Testing:**
```bash
# Run the MCP server
uv run mcp-ai-hub
# Run with custom configuration
uv run mcp-ai-hub --config ./custom_config.yaml --log-level DEBUG
# Run with different transport
uv run mcp-ai-hub --transport sse --port 3001
# Run tests (when test suite is added)
uv run pytest
# Run tests with coverage
uv run pytest --cov=src/mcp_ai_hub --cov-report=html
```
**Code Quality:**
```bash
# Format code with ruff
uv run ruff format .
# Lint code
uv run ruff check .
# Type checking with mypy
uv run mypy src/
# Run all quality checks
uv run ruff format . && uv run ruff check . && uv run mypy src/
```
## Troubleshooting
### Configuration Issues
**Configuration File Problems:**
- **File Location**: Ensure `~/.ai_hub.yaml` exists in your home directory
- **YAML Validity**: Validate YAML syntax using online validators or `python -c "import yaml; yaml.safe_load(open('~/.ai_hub.yaml'))"`
- **File Permissions**: Set secure permissions with `chmod 600 ~/.ai_hub.yaml`
- **Path Resolution**: Use absolute paths in custom config locations
**Configuration Validation:**
- **Required Fields**: Each model must have `model_name` and `litellm_params`
- **API Keys**: Verify API keys are properly quoted and not expired
- **Model Formats**: Use LiteLM-compatible model identifiers (e.g., `openai/gpt-4`, `anthropic/claude-3-5-sonnet-20241022`)
### API and Authentication Errors
**Authentication Issues:**
- **Invalid API Keys**: Check for typos, extra spaces, or expired keys
- **Insufficient Permissions**: Verify API keys have necessary model access permissions
- **Rate Limiting**: Monitor API usage and implement retry logic if needed
- **Regional Restrictions**: Some models may not be available in all regions
**API-Specific Troubleshooting:**
- **OpenAI**: Check organization settings and model availability
- **Anthropic**: Verify Claude model access and usage limits
- **Azure OpenAI**: Ensure proper resource deployment and endpoint configuration
- **Google Gemini**: Check project setup and API enablement
### MCP Connection Issues
**Server Startup Problems:**
- **Port Conflicts**: Use different ports for SSE/HTTP transports if defaults are in use
- **Permission Errors**: Ensure executable permissions for `mcp-ai-hub` command
- **Python Path**: Verify Python environment and package installation
**Client Configuration Issues:**
- **Command Path**: Ensure `mcp-ai-hub` is in PATH or use full absolute path
- **Working Directory**: Some MCP clients require specific working directory settings
- **Transport Mismatch**: Use stdio transport for Claude Desktop/Code
### Performance and Reliability
**Response Time Issues:**
- **Network Latency**: Use geographically closer API endpoints when possible
- **Model Selection**: Some models are faster than others (e.g., GPT-3.5 vs GPT-4)
- **Token Limits**: Large `max_tokens` values can increase response time
**Reliability Improvements:**
- **Retry Logic**: Implement exponential backoff for transient failures
- **Timeout Configuration**: Set appropriate timeouts for your use case
- **Health Checks**: Monitor server status and restart if needed
- **Load Balancing**: Use multiple model configurations for redundancy
## License
MIT License - see [LICENSE](LICENSE) file for details.
## Contributing
We welcome contributions! Please follow these guidelines:
### Development Workflow
1. **Fork and Clone**: Fork the repository and clone your fork
2. **Create Branch**: Create a feature branch (`git checkout -b feature/amazing-feature`)
3. **Development Setup**: Install dependencies with `uv sync`
4. **Make Changes**: Implement your feature or fix
5. **Testing**: Add tests and ensure all tests pass
6. **Code Quality**: Run formatting, linting, and type checking
7. **Documentation**: Update documentation if needed
8. **Submit PR**: Create a pull request with detailed description
### Code Standards
**Python Style:**
- Follow PEP 8 style guidelines
- Use type hints for all functions
- Add docstrings for public functions and classes
- Keep functions focused and small
**Testing Requirements:**
- Write tests for new functionality
- Ensure existing tests continue to pass
- Aim for good test coverage
- Test edge cases and error conditions
**Documentation:**
- Update README.md for user-facing changes
- Add inline comments for complex logic
- Update configuration examples if needed
- Document breaking changes clearly
### Quality Checks
Before submitting a PR, ensure:
```bash
# All tests pass
uv run pytest
# Code formatting
uv run ruff format .
# Linting passes
uv run ruff check .
# Type checking passes
uv run mypy src/
# Documentation is up to date
# Configuration examples are valid
```
### Issues and Feature Requests
- Use GitHub Issues for bug reports and feature requests
- Provide detailed reproduction steps for bugs
- Include configuration examples when relevant
- Check existing issues before creating new ones
- Label issues appropriately