Support Agent + MCP

Live demo: support-agent-mcp.onrender.com/docs — interactive Swagger UI. Health: /healthz. Hosted on Render's free tier, which sleeps after ~15 min idle, so the first request may take 30–60s to wake.

A production-shaped customer-support AI agent: a FastAPI service where a LangGraph agent answers customer questions by calling tools — order lookup, OAuth-scoped refunds, and a grounded knowledge base — with token streaming, conversation memory, OpenTelemetry traces and structured JSON logs. The same tools are re-exposed over the Model Context Protocol (MCP) for any MCP client.

Runs entirely on free infrastructure (Gemini free tier, a free Render/Spaces dyno) and the whole test suite runs offline, with no API key.

Flagship features → what they demonstrate

What makes this production-shaped

It is not the feature list — it is the failure behaviour:

• Authorization is enforced at the tool, not in the prompt. The model can decide to refund; without refund:write on the caller's token the tool still refuses, and the denial is recorded as a span attribute for audit.

• Degrades instead of dying. No API key → /healthz and /token still serve and chat returns an actionable 503. No vector service → the keyword retriever takes over. No collector → traces go to stdout. A broken stream closes with an error event rather than a half-written response.

• Every answer is attributable. Policy replies carry citations recovered from the retriever's own output, and every log line carries the request id also returned in X-Request-ID.

• The offline/online split is deliberate. Pure logic (SSE translation, turn slicing, eval graders, exporter selection) is separated from I/O, so CI proves behaviour with no network and no key — and the same graders score a live Gemini run locally.

Related MCP server: Relay

Architecture

flowchart LR
    subgraph Client
      U[HTTP client / UI]
      MCPC[MCP client<br/>Claude, IDEs]
    end

    U -->|POST /chat<br/>POST /chat/stream<br/>Bearer token| API[FastAPI<br/>JSON logs + request id]
    API -->|JWT scopes| A["LangGraph ReAct agent<br/>(Gemini)"]
    A <-->|thread_id| M[(MemorySaver<br/>checkpointer)]
    A --> T1[get_order_status]
    A --> T2["request_refund<br/>needs refund:write"]
    A --> T3[search_kb]
    T3 --> KB[(KB: keyword / Chroma)]
    API -->|reply · tool_calls · citations<br/>or SSE token stream| U
    MCPC --> S[FastMCP server] --> T1 & T3

    A -.spans.-> OTEL{{OpenTelemetry<br/>console / OTLP}}
    T1 & T2 & T3 -.spans.-> OTEL

Quickstart

python -m venv .venv && source .venv/bin/activate
pip install -r requirements-dev.txt
cp .env.example .env          # add a free key from https://aistudio.google.com/apikey
pytest -q                     # runs fully offline, no key needed
uvicorn app.main:app --reload

Or with Docker:

docker compose up --build     # spans stream to the container log

Interactive API docs at http://localhost:8000/docs.

Demo

# 1) Authenticate as an agent (gets refund:write). Try 'customer/customer' to see a denial.
TOKEN=$(curl -s -X POST localhost:8000/token -d 'username=agent&password=agent' | python -c 'import sys,json;print(json.load(sys.stdin)["access_token"])')

# 2) Grounded policy answer (returns citations)
curl -s -X POST localhost:8000/chat -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' -d '{"message":"How long do refunds take?"}'

# 3) Authorized action
curl -s -X POST localhost:8000/chat -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' -d '{"message":"Refund order A1001, it was defective."}'

# 4) Streamed answer — tokens and tool steps as they happen
curl -N -X POST localhost:8000/chat/stream -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' -d '{"message":"How long do refunds take?"}'

# 5) Multi-turn memory — reuse the session_id and the agent remembers the order
curl -s -X POST localhost:8000/chat -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' -d '{"message":"Where is order A1002?","session_id":"demo-1"}'
curl -s -X POST localhost:8000/chat -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' -d '{"message":"Can I refund it?","session_id":"demo-1"}'

Endpoints

Observability

Tracing is controlled by OTEL_TRACES_EXPORTER: console (default — spans to stdout, zero infrastructure), otlp (any OTLP/HTTP collector via OTEL_EXPORTER_OTLP_ENDPOINT), or none. Spans cover the agent run, each tool call and the LLM call:

{"name": "tool.request_refund", "attributes": {"tool.name": "request_refund", "authz.allowed": false}}

Logs are one JSON object per line, each carrying the request_id that is also returned in the X-Request-ID response header (LOG_FORMAT=text for a human-readable dev view):

{"ts": "…", "level": "INFO", "logger": "support-agent.access", "message": "request",
 "request_id": "9e8d65214aae4af4", "method": "GET", "path": "/healthz", "status": 200, "duration_ms": 0.75}

Evals

evals/ scores the three behaviours the agent is actually hired for: picking the right tool, refusing an unauthorized refund, and answering policy questions from the KB with a citation.

python -m evals.runner --min-pass-rate 0.8   # live Gemini run; exits non-zero below the bar

The graders (evals/scorers.py) are pure functions, so CI exercises them against fixtures with no key; the live end-to-end run is pytest.mark.skipif-ed off when GOOGLE_API_KEY is unset. That is what keeps CI hermetic while the same rubric grades a real model locally.

MCP

python -m mcp_server.server   # exposes order_status + knowledge_base over stdio

Refunds are intentionally not exposed over MCP: that action requires a scope-bearing session, which the local stdio transport does not carry.

Deploy (free tiers)

Render — render.yaml is a ready blueprint (free plan, Docker runtime, health check on /healthz). Push the repo, then Render → New → Blueprint → select it. GOOGLE_API_KEY is declared sync: false, so Render prompts for it in the dashboard and it never enters git; JWT_SECRET is generated per environment. Free instances sleep when idle, so the first request after a nap is slow.

Hugging Face Spaces — deploy/huggingface/ holds a Spaces-ready Dockerfile (port 7860, non-root user) and the Space README.md with the required front matter. Copy both to a Docker Space along with requirements.txt, app/, mcp_server/ and evals/, then add GOOGLE_API_KEY under Settings → Variables and secrets. Step-by-step: deploy/huggingface/README.md.

Secrets are always injected as environment variables. .env is gitignored; .env.example documents every variable.

Configuration

Tech

Python · FastAPI · Pydantic · LangChain · LangGraph · MCP · SSE · OAuth2/JWT · OpenTelemetry · Chroma/pgvector · Docker · GitHub Actions