Skip to main content
Glama

Cortex

An MCP-native agentic platform: planner/executor/critic over hybrid RAG, with the budget, safety and observability layers a production deployment needs.

241 tests · 82% coverage, gate-enforced · runs offline against fakes · no service required to develop

Cortex combines MCP, multi-agent orchestration, hybrid RAG, three-tier memory, LLMOps, automated evaluation and safety guardrails in one codebase. What it is not is a system that has been run at scale: it has never served production traffic, and the load-testing and scale-out work is listed in LIMITATIONS.md rather than implied here.


Why Cortex

Most AI projects are tutorials. Cortex is built the way a senior AI engineer would build it for production:

  • MCP-native — all capabilities exposed as Model Context Protocol tools. Connect Claude Desktop, VS Code, or any MCP client to your running Cortex server in minutes.

  • Multi-LLM via LiteLLM — swap between OpenAI, Anthropic, Azure, Bedrock, or Vertex by changing one config value.

  • Full observability — every LLM call, agent step, and tool invocation is an OTEL span, visible in Arize Phoenix and Grafana.

  • Automated evaluation — Ragas metrics with a regression suite on a 6-hourly schedule. Faithfulness is a tracked metric with a threshold, not a hope.

  • Budget enforcement — a per-run cost ceiling checked before every model call, against a Redis ledger shared across async tasks and workers.

  • Rate limiting that runs before routing — so requests that 404 or 422 are metered too. Those are cheaper for an attacker to generate than valid ones.


Related MCP server: Agentic Control Framework (ACF)

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                         MCP Clients                                  │
│         Claude Desktop │ VS Code │ Custom Agents │ REST API          │
└─────────────────┬───────────────────────────────────────────────────┘
                  │ MCP / HTTP
┌─────────────────▼───────────────────────────────────────────────────┐
│                    Cortex API Gateway (FastAPI)                        │
│              Auth │ Rate Limiting │ SSE Streaming │ /metrics          │
└─────────────────┬───────────────────────────────────────────────────┘
                  │
┌─────────────────▼───────────────────────────────────────────────────┐
│                 LangGraph Agent Orchestration                         │
│        load_memory → planner → executor → critic → save_memory       │
└──────┬──────────┬──────────────┬──────────────┬───────────────────-─┘
       │          │              │              │
  ┌────▼───┐ ┌───▼────┐ ┌──────▼────┐ ┌──────▼──────┐
  │  MCP   │ │  RAG   │ │  Memory  │ │   Safety    │
  │ Server │ │Pipeline│ │  3-Tier  │ │ Guardrails  │
  └────┬───┘ └───┬────┘ └──────┬────┘ └─────────────┘
       │          │              │
┌──────▼──────────▼──────────────▼──────────────────────────────────-─┐
│                      LiteLLM Router                                   │
│         OpenAI │ Anthropic │ Azure │ Bedrock │ Vertex │ Ollama        │
└───────────────────────────────────────────────────────────────────-──┘
       │          │              │
  ┌────▼──┐  ┌───▼───┐  ┌──────▼──────────────────────────────────┐
  │Qdrant │  │ Redis │  │  Observability: OTEL + Phoenix + Grafana │
  └───────┘  └───────┘  └─────────────────────────────────────────┘

Quickstart (5 minutes)

Prerequisites

  • Python 3.11+

  • Docker + Docker Compose

  • At least one LLM API key (OpenAI recommended for quickstart)

1. Clone and configure

git clone https://github.com/your-org/cortex
cd cortex
cp .env.example .env
# Edit .env — set OPENAI_API_KEY and SECRET_KEY at minimum

2. Start all services

docker compose up -d

Services will be available at:

Service

URL

Cortex API

http://localhost:8000/docs

Cortex MCP

stdio (for Claude Desktop)

Arize Phoenix

http://localhost:6006

Grafana

http://localhost:3000 (admin / cortex)

Qdrant UI

http://localhost:6333/dashboard

Prometheus

http://localhost:9090

3. Ingest a document

curl -X POST http://localhost:8000/api/v1/ingest \
  -H "Authorization: Bearer $(python scripts/gen_token.py)" \
  -H "Content-Type: application/json" \
  -d '{"text": "Your document content here", "metadata": {"source": "quickstart"}}'

4. Run an agent

curl -X POST http://localhost:8000/api/v1/runs \
  -H "Authorization: Bearer $(python scripts/gen_token.py)" \
  -H "Content-Type: application/json" \
  -d '{"goal": "Summarise the key points from the ingested documents"}'

5. Connect Claude Desktop

Add to your Claude Desktop claude_desktop_config.json:

{
  "mcpServers": {
    "cortex": {
      "command": "python",
      "args": ["-m", "cortex.mcp.server"],
      "cwd": "/path/to/cortex"
    }
  }
}

Project Structure

cortex/
├── src/cortex/
│   ├── config.py              # Pydantic Settings — all configuration
│   ├── exceptions.py          # Domain exception hierarchy
│   ├── logging_config.py      # Structlog structured logging
│   ├── mcp/
│   │   ├── server.py          # FastMCP server — MCP tool definitions
│   │   └── client.py          # MCP client for agent tool calls
│   ├── llm/
│   │   ├── router.py          # LiteLLM wrapper with retry, fallback, budget
│   │   ├── cost_tracker.py    # Per-run token cost accounting (Redis)
│   │   └── cache.py           # Semantic cache (Qdrant)
│   ├── graph/
│   │   ├── state.py           # LangGraph state schema (CortexState)
│   │   └── cortex_graph.py     # Graph topology, nodes, routing functions
│   ├── agents/
│   │   ├── planner.py         # Goal → task list decomposition
│   │   ├── executor.py        # Task execution via MCP tools
│   │   ├── critic.py          # Output quality evaluation
│   │   └── memory_agent.py    # Memory retrieval and consolidation
│   ├── rag/
│   │   └── pipeline.py        # Ingest, chunk, embed, hybrid search, rerank
│   ├── memory/                # Three-tier memory (working / episodic / semantic)
│   ├── safety/
│   │   └── middleware.py      # Guardrails, PII scanner, injection detector
│   ├── obs/
│   │   └── metrics.py         # Prometheus metrics + OTEL + Phoenix setup
│   ├── eval/
│   │   └── ragas_runner.py    # Automated Ragas evaluation suite
│   └── api/
│       ├── main.py            # FastAPI app — all HTTP endpoints
│       └── auth.py            # JWT authentication
├── config/
│   └── rails/                 # NeMo Guardrails Colang policies
├── obs/
│   ├── prometheus.yml         # Prometheus scrape config
│   └── grafana/               # Grafana dashboard JSON + provisioning
├── tests/                     # Pytest test suite (80%+ coverage required)
├── docs/adr/                  # Architecture Decision Records
├── docker-compose.yml         # Full local stack
├── Dockerfile                 # Multi-stage production image
├── pyproject.toml             # Dependencies and tooling config
└── .env.example               # All required environment variables

Documentation

Document

What it covers

ARCHITECTURE.md

Full system design, data flows, design decisions

MCP.md

MCP server, tool definitions, connecting clients

AGENTS.md

Agent design, prompts, tool access, failure modes

RAG.md

Ingestion, chunking strategy, hybrid search, evaluation

MEMORY.md

Three-tier memory design and retrieval strategy

LLMOPS.md

Observability setup, metrics catalogue, dashboards

GUARDRAILS.md

Safety policies, PII handling, injection defence

EVALUATION.md

Ragas metrics, regression testing, baseline scores

COST.md

Token tracking, caching, budget enforcement

DEPLOYMENT.md

Docker Compose, Azure/AWS cloud deployment

TESTING.md

Test strategy, coverage targets, eval tests

ADR/001

Why MCP over custom REST for tool exposure

ADR/002

State schema design decisions

ADR/003

Multi-LLM routing strategy

ADR/004

Memory architecture


Running Tests

pip install -e ".[dev]"
pytest                    # Full suite with coverage
pytest tests/test_rag/    # RAG tests only
pytest -k "test_planner"  # Single test

Coverage target: 80% minimum (enforced in CI).


Contributing

See CONTRIBUTING.md. All PRs require:

  • Tests for new code

  • Updated ADR if architecture changes

  • Eval regression suite must still pass (pytest tests/eval/)


What was wrong with this codebase, and how it was found

Cortex was generated complete — 77 files, eight layers, an 80% coverage gate — and had never been executed. Not once. Every defect below was found by making it run.

Severity

Defect

Why nothing caught it

Blocking

The test suite could not be collected. Sixteen modules called get_settings() at import time; SECRET_KEY is unset on a clean machine, so import cortex.obs.metrics raised ValidationError before any test started.

--cov-fail-under=80 was gating a suite that had never run.

Blocking

FastMCP(version=...) is not a valid API. Two different classes share the name; the code imported the one from the mcp SDK and passed it the other one's arguments.

Both packages are declared in pyproject.toml, which made the mistake look plausible.

Blocking

The executor never passed its tool schemas to the model. It fetched them into a local variable and dropped them.

Under a mocked router the loop still "worked" — the mock returns tool calls whether or not any tools were offered. Ruff found it, as an unused variable.

Blocking

Four tests required a live Redis. The mock_router fixture patched cortex.llm.router.get_router, but agents bind that name at import, so the patch rebound a name nobody read.

The fixture looked correct and was a no-op.

High

Rate limiting did not exist. api_rate_limit_per_minute was configured, documented, and read by nothing.

A control an operator believes they have.

High

query_data was a stub advertising itself as a working tool. It returned an empty result set with a note, so an agent would "query the data", get zero rows, and confidently report no matching records.

A tool that fakes success is worse than an absent one — the absent one can be planned around.

High

tenant_id was carried in state, described in the docs, and used by neither memory tier.

Safe only while user ids are globally unique, which they are not when they come from tenant-local identity providers.

High

A Gauge labelled by run_id. Every run minted a permanent time series.

The classic cardinality mistake: invisible until Prometheus falls over weeks later, during an unrelated incident.

Medium

The graph's cost guard read >= 2.0 instead of the configured ceiling — which happens to equal the default, so it looked right and ignored its own configuration.

Medium

A memory-write failure crashed a run that had already produced its answer.

Losing the memory write is a degradation. Losing the answer is a bug.

Medium

Document.id was uuid4() next to an unused doc_hash property, so re-ingesting an unchanged file duplicated the corpus instead of upserting over it.

Medium

An unexpected exception type skipped the LLM fallback entirely and propagated raw, so the documented failure mode was not the one callers got.

Low

Six dependencies declared and never imported; a Postgres service nothing connected to; MemoryError shadowing the builtin.

The reason this table is here rather than quietly fixed: the gap between "generated" and "working" is where all the engineering lives, and every row is a question I can answer in depth.

A
license - permissive license
-
quality - not tested
B
maintenance

Maintenance

Maintainers
Response time
Release cycle
Releases (12mo)
Commit activity

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Servers

  • A
    license
    -
    quality
    C
    maintenance
    An advanced MCP-based AI agent system with intelligent tool orchestration, multi-LLM support, and enterprise-grade reliability features like semantic routing and circuit breakers.
    MIT
  • A
    license
    -
    quality
    B
    maintenance
    Enables running durable, traceable AI agents via LangGraph through a universal MCP interface, integrating with Hatchet for orchestration, logging, and retries. Provides tools for knowledge management (ingestion, RAG) and Kubernetes operations (diagnosis, auto-fix).
    MIT

View all related MCP servers

Related MCP Connectors

  • Control plane for autonomous software labor. Agents claim objectives over MCP with audit trail.

  • OCR, transcription, file extraction, and image generation for AI agents via MCP.

  • User-owned memory for AI agents, Copilot, Claude, IDEs, CLIs, and chat apps over remote MCP.

View all MCP Connectors

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/mrinmoyece/cortex'

If you have feedback or need assistance with the MCP directory API, please join our Discord server