📊 Autonomous Data Analysis System (MCP)

An AI data analyst that answers natural-language questions over CSV / Excel / SQL data. The language model plans and interprets — all computation (SQL, stats, cleaning, charts) runs in deterministic MCP tools, so results are exact, reproducible, and auditable.

Core principle: the LLM never does arithmetic. It reads schemas, chooses which tool to call with which arguments, and turns the returned numbers into business insight.

Architecture

┌──────────────────────────────────────────────────────────┐
│  Streamlit UI   (upload · chat · charts · report · switch) │
└───────────────┬──────────────────────────────────────────┘
                │
┌───────────────▼──────────────────────────────────────────┐
│  Agent host                                                │
│   • Provider switch (LiteLLM): Ollama · Groq · Gemini      │
│   • MCP client: MCP tool schemas ⇆ OpenAI function calls   │
│   • Loop: plan → call tool → observe → answer              │
└───────────────┬──────────────────────────────────────────┘
                │ MCP protocol (stdio)
┌───────────────▼──────────────────────────────────────────┐
│  MCP analysis server                                       │
│   load · profile · sql · eda · clean · stats · viz · report│
│   Dataset registry · Pandas · DuckDB · SciPy · matplotlib  │
└──────────────────────────────────────────────────────────┘

See DESIGN.md for the full design and milestone tracker.

Related MCP server: xtai-mcp-data-analysis

Features

• Upload CSV / Excel; DuckDB SQL over your files with no database server.

• Automated profiling & EDA — schema, nulls, cardinality, correlations, group aggregates.

• Non-destructive cleaning — missing values, duplicates, type casts, outliers (each returns a new versioned dataset).

• Statistics — t-test / ANOVA / chi-square, correlation tests, trend analysis, distribution/normality.

• Charts — bar, line, scatter, histogram, box, correlation heatmap (PNG).

• Self-contained HTML reports with embedded charts.

• Pluggable LLM — local Ollama by default, switch to Groq or Gemini free tiers.

Setup

python3 -m venv .venv
source .venv/bin/activate          # fish: source .venv/bin/activate.fish
pip install -r requirements.txt
cp .env.example .env               # then fill in the provider you want

Choosing a provider (edit .env)

You can also switch provider live from the sidebar dropdown in the UI.

Run the app

streamlit run ui/streamlit_app.py

Then upload one of the samples in data/ (e.g. sample_sales.csv) and ask things like "Which region sells the most units, and is the trend rising?" or "Which plan has the highest churn — build me a report."

Use the MCP server directly (e.g. Claude Desktop)

The server speaks stdio and works with any MCP client:

python -m mcp_server.server

Claude Desktop config:

{ "mcpServers": {
    "data-analysis": { "command": "python", "args": ["-m", "mcp_server.server"] }
} }

MCP tool catalog

Testing

PYTHONPATH="$PWD" pytest -q

15 tests spawn the real MCP server over stdio and exercise every tool group; the agent loop is tested with a scripted fake LLM (deterministic, no network).

Project structure

mcp_server/     FastMCP server + registry + tools/
agent/          provider switch (config, providers) + mcp_client + agent loop
ui/             streamlit_app.py
data/           sample datasets + uploads/
reports/        generated charts and HTML reports
tests/          per-milestone test suites