# Multi-Cloud LLM Platform

A provider-agnostic LLM routing layer spanning AWS Bedrock, OpenAI, and local Ollama models, instrumented with Prometheus metrics, wrapped in a LangGraph iterative research agent, and exposed to any MCP-compatible client (e.g. Claude Desktop) as callable tools over an MCP stdio server.

## Overview

```
              ┌─────────────── src/providers.py ───────────────┐
              │  claude-sonnet-bedrock   (AWS Bedrock)          │
prompt ──────▶│  llama3-bedrock          (AWS Bedrock)          │──▶ LLMResponse
   or          │  gemini-flash-vertex     (low-cost tier)        │    (+ Prometheus metrics:
route(task) ─▶│  llama3-local            (Ollama, local/free)   │     requests, latency, cost, tokens)
              └──────────────────────────────────────────────────┘
                          ▲                        ▲
                          │                        │
              agents/research_agent.py      mcp_server/server.py
              (LangGraph loop: gather        (exposes generate/route/
               → evaluate → write report)     list_providers as MCP tools)
```

`route(prompt, task_type)` picks a provider based on task type (`code`/`reason` → Claude Sonnet on Bedrock, `summarize` → Llama 3 on Bedrock, `low-cost` → the low-cost tier, `general` → local Ollama), so callers don't need to know which backend is cheapest or best suited for a given job.

## Project structure

```
.
├── src/
│   ├── providers.py       # Provider catalog + generate()/route(), Prometheus instrumentation
│   └── metrics.py         # Prometheus Counter/Histogram definitions
├── agents/
│   └── research_agent.py  # LangGraph agent: iteratively researches a topic, then writes a report
├── mcp_server/
│   └── server.py          # MCP stdio server exposing generate/route/list_providers as tools
├── tests/
│   └── test_platform.py   # Provider registration, routing table, and research-agent tests
├── run_mcp.sh              # Convenience launcher for the MCP server
└── requirements.txt
```

## Providers

| Key                   | Backend                                  | Cost / 1K tokens | Strengths                       |
|-----------------------|-------------------------------------------|------------------|----------------------------------|
| `claude-sonnet-bedrock` | Claude 3.5 Sonnet via AWS Bedrock        | $0.003           | reasoning, writing, code         |
| `llama3-bedrock`        | Llama 3 8B Instruct via AWS Bedrock      | $0.0003          | summarization, classification    |
| `gemini-flash-vertex`   | Gemini 1.5 Flash                          | $0.0005          | low-cost, fast, general          |
| `llama3-local`          | Llama 3 via local Ollama                 | $0.0              | privacy, offline, no-cost        |

Every call increments Prometheus counters/histograms for request count, latency, estimated cost, and token usage, labeled by provider.

## Research agent

`agents/research_agent.py` builds a small LangGraph loop:

1. **gather_information** — asks the model (local Llama 3 by default) for 3–5 new facts on the topic not already covered.
2. **evaluate_sufficiency** — asks the model whether enough has been gathered for a 3-paragraph report.
3. Loops back to step 1 (up to 3 iterations) or proceeds to **write_report**, which synthesizes all notes into the final report.

## MCP server

`mcp_server/server.py` runs an MCP stdio server (`multi-cloud-llm-platform`) that advertises three tools — `generate`, `route`, and `list_providers` — so any MCP client can call these LLM providers directly. Launch it with:

```bash
python -m mcp_server.server
```

## Setup

```bash
python -m venv venv && source venv/bin/activate
pip install -r requirements.txt
```

- **AWS Bedrock** (`claude-sonnet-bedrock`, `llama3-bedrock`): configure AWS credentials (`~/.aws/credentials`, env vars, or an IAM role) with Bedrock model access enabled in `us-east-1` for the Claude 3.5 Sonnet and Llama 3 models.
- **Low-cost tier** (`gemini-flash-vertex`): requires `export GOOGLE_API_KEY=<your-google-ai-studio-key>`.
- **Local** (`llama3-local`): install [Ollama](https://ollama.com) and run `ollama pull llama3`.

## Usage

```bash
# Run a single generation or routed call
python -c "from src.providers import route; print(route('Summarize this quarter', task_type='summarize').text)"

# Run the iterative research agent
python -m agents.research_agent

# Run the MCP server
./run_mcp.sh   # or: python -m mcp_server.server

# Run tests
pytest
```

## Tech stack

LangChain, LangGraph, AWS Bedrock (boto3), Google Gemini, Ollama, MCP (`mcp`), Prometheus client, FastAPI/uvicorn, pytest.