karenina-mcp

Experimental - This is an experimental MCP server for inspecting Karenina verification results through natural language queries.

Overview

karenina-mcp provides an MCP (Model Context Protocol) interface that allows AI assistants like Claude to explore and analyze verification results stored in a Karenina SQLite database. Instead of writing SQL queries manually, you can ask questions in natural language and the assistant will translate them into appropriate queries.

Related MCP server: MCP SQL Agent

How It Works

The server uses a hierarchical context exposition approach to help the assistant understand your database efficiently:

Step 1: Configure the Database

First, call configure_database with the path to your SQLite results database. This connects the server and returns a list of available tables and views.

Step 2: Query with Natural Language

Once configured, the agent uses hierarchical schema discovery to answer your questions:

1. Schema Awareness - View summaries are embedded in the get_schema tool description, so the agent sees all available views without any tool call

2. Selective Deep-Dive - The agent calls get_schema([view_names]) only for views relevant to your question

3. Query Generation - With precise schema knowledge, it generates accurate SQL queries

4. Results Interpretation - Results are returned as formatted markdown tables

This approach minimizes context usage while ensuring the assistant has the precise information needed to answer your questions accurately.

┌─────────────────────────────────────────────────────────────────┐
│               configure_database(db_path)                        │
│   Points the server to the SQLite results database               │
│   → Returns list of available tables and views                   │
└─────────────────────────────────────────────────────────────────┘
                                │
                    (database now connected)
                                │
                                ▼
┌─────────────────────────────────────────────────────────────────┐
│                     User Question                                │
│            "Which model performed best on biology questions?"    │
└─────────────────────────────────────────────────────────────────┘
                                │
                                ▼
┌─────────────────────────────────────────────────────────────────┐
│          Agent reads tool descriptions (no call needed)          │
│   get_schema description contains one-line view summaries        │
│   → Agent identifies relevant views for the question             │
└─────────────────────────────────────────────────────────────────┘
                                │
                                ▼
┌─────────────────────────────────────────────────────────────────┐
│           get_schema(["template_results", ...])                  │
│   Returns full column docs, types, keys, joins, examples         │
│   → Agent now knows exact column names and relationships         │
└─────────────────────────────────────────────────────────────────┘
                                │
                                ▼
┌─────────────────────────────────────────────────────────────────┐
│                        query(sql)                                │
│   Agent generates precise SQL with correct column names          │
│   → Returns formatted markdown table with results                │
└─────────────────────────────────────────────────────────────────┘

Installation

cd karenina-mcp
uv sync

Usage

Run the server (STDIO mode)

uv run karenina-mcp
# or
uv run fastmcp run src/karenina_mcp/server.py

Run as HTTP server

Start the MCP server as an HTTP server for remote or web-based access:

uv run fastmcp run src/karenina_mcp/server.py --transport http --port 8000

The server will be available at http://localhost:8000. You can also specify a custom host:

uv run fastmcp run src/karenina_mcp/server.py --transport http --host 0.0.0.0 --port 8000

Configure in Claude Code

Add to your Claude Code settings (.claude/settings.local.json or global settings):

{
  "mcpServers": {
    "karenina": {
      "command": "uv",
      "args": ["--directory", "/path/to/karenina-mcp", "run", "karenina-mcp"]
    }
  }
}

Replace /path/to/karenina-mcp with the absolute path to the karenina-mcp directory.

Configure in Claude Desktop

Add to your Claude Desktop config (~/Library/Application Support/Claude/claude_desktop_config.json):

{
  "mcpServers": {
    "karenina": {
      "command": "uv",
      "args": ["--directory", "/path/to/karenina-mcp", "run", "karenina-mcp"]
    }
  }
}

Tools

configure_database

Initialize the server with your results database.

configure_database(db_path="/path/to/karenina.db")

Returns confirmation with list of available tables and views.

get_schema

Get detailed schema documentation for specific views. The tool description itself contains one-line summaries of all available views, so the agent can identify relevant views without calling the tool.

get_schema(view_names=["template_results", "question_attributes"])

Returns full column documentation, types, primary/foreign keys, join information, and example queries for the requested views.

Example Questions

Once the database is configured, you can ask questions like:

• "What's the overall pass rate across all models?"

• "Show me the questions where "mcp-local" was correct but "mcp-remote" failed;

• "Compute pass rates by question keywords and sort them in increasing performance"

• Show me results to question from the last run where more than one but not all of the replicates failed;

Related Projects

• Karenina - Core benchmarking framework