Iris — MCP-Native Agent Eval & Observability

Iris is an open-source Model Context Protocol (MCP) server that provides trace logging, quality evaluation, and cost tracking for AI agents. Any MCP-compatible agent framework can discover and invoke Iris tools.

Quickstart

npm install -g @iris-eval/mcp-server
iris-mcp

Or run directly:

npx @iris-eval/mcp-server

Docker

docker run -p 3000:3000 -v iris-data:/data ghcr.io/iris-eval/mcp-server

Configuration

Iris looks for config in this order (later overrides earlier):

1. Built-in defaults

2. ~/.iris/config.json

3. Environment variables (IRIS_*)

4. CLI arguments

CLI Arguments

Environment Variables

Security

When using the HTTP transport, Iris includes production-grade security:

• Authentication — Set IRIS_API_KEY or --api-key to require Authorization: Bearer <key> on all endpoints (except /health). Recommended for any network-exposed deployment.

• CORS — Restricted to http://localhost:* by default. Configure with IRIS_ALLOWED_ORIGINS.

• Rate limiting — 100 requests/minute for dashboard API, 20 requests/minute for MCP endpoints. Configurable via ~/.iris/config.json.

• Security headers — Helmet middleware applies CSP, X-Frame-Options, X-Content-Type-Options, and other standard headers.

• Input validation — All query parameters validated with Zod schemas. Malformed requests return 400.

• Request size limits — Body payloads limited to 1MB by default.

• Safe regex — User-supplied regex patterns in custom eval rules are validated against ReDoS attacks.

• Structured logging — JSON logs to stderr via pino. Never writes to stdout (reserved for stdio transport).

# Production deployment example
iris-mcp --transport http --port 3000 --api-key "$(openssl rand -hex 32)" --dashboard

MCP Tools

log_trace

Log an agent execution trace with spans, tool calls, and metrics.

Input:

• agent_name (required) — Name of the agent

• input — Agent input text

• output — Agent output text

• tool_calls — Array of tool call records

• latency_ms — Execution time in milliseconds

• token_usage — { prompt_tokens, completion_tokens, total_tokens }

• cost_usd — Total cost in USD

• metadata — Arbitrary key-value metadata

• spans — Array of span objects for detailed tracing

evaluate_output

Evaluate agent output quality using configurable rules.

Input:

• output (required) — The text to evaluate

• eval_type — Type: completeness, relevance, safety, cost, custom

• expected — Expected output for comparison

• trace_id — Link evaluation to a trace

• custom_rules — Array of custom rule definitions

get_traces

Query stored traces with filters and pagination.

Input:

• agent_name — Filter by agent name

• framework — Filter by framework

• since — ISO timestamp lower bound

• until — ISO timestamp upper bound

• min_score / max_score — Score range filter

• limit — Results per page (default 50)

• offset — Pagination offset

MCP Resources

• iris://dashboard/summary — Dashboard summary statistics

• iris://traces/{trace_id} — Full trace detail with spans and evals

Claude Desktop

Add Iris to your Claude Desktop MCP config:

{
  "mcpServers": {
    "iris-eval": {
      "command": "npx",
      "args": ["@iris-eval/mcp-server"]
    }
  }
}

Then ask Claude to "log a trace" or "evaluate this output" — Iris tools are automatically available.

See examples/claude-desktop/ for more configuration options.

Web Dashboard

Start with --dashboard flag to enable the web UI at http://localhost:6920.

Examples

• Claude Desktop setup — MCP config for stdio and HTTP modes

• TypeScript — MCP SDK client usage

• LangChain — Agent instrumentation

• CrewAI — Crew observability

Community

• GitHub Issues — Bug reports and feature requests

• GitHub Discussions — Questions and ideas

• Contributing Guide — How to contribute

• Roadmap — What's coming next

License

MIT