# Climate MCP Server

A [Model Context Protocol (MCP)](https://modelcontextprotocol.io) server that exposes 26 structured analytical tools over Climate Finance Update (CFU) datasets. Designed for use with Claude Desktop (STDIO) or any MCP-compatible client.

## Overview

The server enables structured queries against three Climate Finance datasets (DuckDB-backed) — funds, pledges, and projects — without requiring the client to write any data access code. All tools return a consistent JSON envelope and emit audit events for full reproducibility.

**Key capabilities:**
- Fund-level financial analysis (stage totals, conversion ratios, gaps)
- Portfolio-wide aggregations by fund type, sector, adaptation/mitigation
- Contributor/donor ranking across pledge and deposit stages
- Data quality diagnostics (missing value reports)
- Schema contract introspection
- Run-level audit trail with Markdown export

---

## Project Report

For a narrative project description (problem statement, architecture, deployment, and validation), see:

- `docs/PROJECT_REPORT.md`

## Project Structure

```
climate_mcp_server/
├── data/                              # Optional compatibility exports (legacy CSV aliases)
│   ├── fund.csv
│   ├── pledges.csv
│   └── projects.csv
│
├── ingestion_layer/                   # CFU download → DuckDB → CSV (runs at MCP startup unless CLIMATE_MCP_SKIP_CFU_PIPELINE=1)
│   ├── cfu_pipeline.py                # Main pipeline: scrape dashboard, download Excel, load DuckDB, export CSVs
│   ├── transform.py                   # Transform / analytics steps used by the pipeline
│   ├── Requirements.txt               # Standalone ingestion dependencies (optional)
│   ├── notification developer.py      # Optional notifications (email, etc.)
│   └── data/
│       ├── climate_funds.duckdb       # Primary store: raw.* + analytics.* tables
│       ├── manifest.json              # Last run: URLs, checksums, paths to exports
│       ├── raw_change_report.json     # Diff-style report between ingested versions
│       ├── transform_schema_review.json
│       ├── archive/                   # Timestamped Excel workbooks (*.xlsx)
│       ├── raw/                       # latest.xlsx + latest.sha256.txt (working copy)
│       ├── raw_snapshots/             # Per-sheet CSV snapshots (*_latest.csv)
│       └── exports/                   # Versioned CSVs for inspection / non-DuckDB fallback
│           ├── fund/fund1.csv
│           ├── pledges/pledges1.csv
│           └── projects/projects1.csv
│
├── docs/
│   ├── schema_contract.md             # Human-readable governed schema contract
│   └── schema_contract_summary.json   # Machine-readable contract summary (LLM validation)
│
├── runs/                              # Auto-generated audit run logs
│   └── *.md                           # One Markdown file per run_id
│
├── mcp_server/
│   ├── finance/
│   │   └── server.py                  # MCP server entry point (FastMCP initialisation)
│   │
│   ├── tools/                         # All 26 MCP tools
│   │   ├── registry.py                # Central tool registration (single entry point)
│   │   ├── context.py                 # Shared singleton context: paths, table loader, audit state, and shared helpers
│   │   ├── audit_tools.py             # start_run / get_audit_trail / finalize_run
│   │   ├── contract_tools.py          # get_schema_contract / get_schema_contract_summary
│   │   ├── discovery_tools.py         # list_datasets / describe_dataset / preview_dataset / list_unique_values / resolve_entity
│   │   ├── summary_tools.py           # search_funds / get_fund_details / fund_summary / portfolio_summary / sector_summary /
│   │   │                              # fund_type_distribution / adaptation_vs_mitigation_summary / compare_funds / fund_stage_totals
│   │   ├── finance_tools.py           # fund_conversion_ratios / rank_entities / rank_by_column / filter_funds_by_threshold / fund_stage_gap
│   │   ├── aggregation_tools.py       # avg_projects_per_fund_by_type
│   │   └── quality_tools.py           # missing_report
│   │
│   └── data/
│       └── datasets.py                # Dataset URI registry (cfu://funds → analytics.fund, etc.)
│
├── mcp_client/                        # MCP client package
│   ├── CLIENT_ARCHITECTURE.md
│   ├── __init__.py
│   ├── __main__.py
│   ├── agent.py
│   ├── api.py
│   ├── config.py
│   ├── erasmus.py
│   ├── errors.py
│   └── mcp_tools.py
│
├── tests/
│   └── run_tests.py                   # Self-contained test runner (111 tests, no pytest required)
│
├── pyproject.toml                     # Project metadata and dependencies
├── uv.lock                            # Locked dependency versions
├── .python-version                    # Python version pin
└── .gitignore


```

---

## Architecture

```
MCP Client (ClimateGPT)
        │
        │  JSON-RPC tool request
        ▼
MCP Server — FastMCP ("finance")
        │
        ├── Tool Layer (7 modules, 26 tools)
        │      audit_tools      → run lifecycle management
        │      contract_tools   → schema governance
        │      discovery_tools  → dataset exploration
        │      summary_tools    → fund profiles and aggregations
        │      finance_tools    → ratios, rankings, gaps, thresholds
        │      aggregation_tools→ cross-dataset aggregations
        │      quality_tools    → data completeness diagnostics
        │
        ├── Shared Context (context.py)
        │      Singleton ToolContext — thread-safe, mtime-cached dataset loading
        │      Shared helpers: _to_num, _records, _load, _norm
        │      Audit state: active_run_id, run_events, log_tool_event()
        │      Uniform output envelope: tool_result()
        │
        ├── Dataset Registry (datasets.py)
        │      URI resolution: cfu://funds → analytics.fund
        │                      cfu://pledges → analytics.pledges
        │                      cfu://projects → analytics.projects
        │
        └── DuckDB Dataset Tables
               analytics.fund · analytics.pledges · analytics.projects
```

**Design principles:**
- Every tool returns the same JSON envelope: `{"ok": bool, "data": {...}, "meta": {...}, "error": null | "message"}`
- Dataset tables are loaded once and cached by DuckDB file mtime; no redundant reads across tools
- All string comparisons are case-insensitive via `_norm()` (lowercase + strip)
- NaN values are replaced with `null` before serialisation via `_records()`
- The singleton context is protected by a double-checked lock for thread safety

---

## Datasets

| URI | Database Table | Legacy Alias | Primary Key | Grain |
|-----|----------------|--------------|-------------|-------|
| `cfu://funds` | `analytics.fund` | `fund.csv` | `fund_id` | One row per fund |
| `cfu://pledges` | `analytics.pledges` | `pledges.csv` | `pledge_id` | One row per pledge record |
| `cfu://projects` | `analytics.projects` | `projects.csv` | `project_id` | One row per project |

**Financial stages** (in pipeline order): `pledge → deposit → approval → disbursement`

---

## Tools Reference

### Audit Tools (3)
| Tool | Description |
|------|-------------|
| `start_run(query)` | Begin a new audit run; clears previous events and returns a `run_id` |
| `get_audit_trail()` | Compact view of tools called in the current run with datasets and columns used |
| `finalize_run(query?, extra_notes?)` | Write the full run log to `runs/<run_id>.md` and clear memory |

### Contract Tools (2)
| Tool | Description |
|------|-------------|
| `get_schema_contract_summary()` | Machine-readable JSON schema contract (call before cross-dataset analysis) |
| `get_schema_contract()` | Full schema contract as Markdown text |

### Discovery Tools (5)
| Tool | Description |
|------|-------------|
| `list_datasets()` | All registered datasets with URI, primary key, grain, and description |
| `describe_dataset(filename?, dataset_uri?)` | Row count, column names, dtypes, and null counts |
| `preview_dataset(n?, filename?, dataset_uri?)` | First N rows (max 25) |
| `list_unique_values(column, limit?, filename?, dataset_uri?)` | Sorted unique non-null values in a column (max 200) |
| `resolve_entity(column, query, top_k?, filename?, dataset_uri?)` | Fuzzy-match a user string against real column values — call before passing names to other tools |

### Summary Tools (9)
| Tool | Description |
|------|-------------|
| `search_funds(query, limit?)` | Keyword search across fund name, type, focus, and sector |
| `get_fund_details(fund_id)` | Full dataset record for a single fund |
| `fund_summary(fund_id)` | Profile card: metadata, stage metrics, portfolio share, and conversion ratios |
| `portfolio_summary()` | Portfolio-wide totals: fund count, four-stage sums, adaptation/mitigation counts |
| `sector_summary(metric?, top_n?)` | Totals and percentage share by `fund_focus_sector` |
| `fund_type_distribution(metric?, top_n?)` | Totals and percentage share by `fund_type` |
| `adaptation_vs_mitigation_summary(metric?)` | Funds bucketed into adaptation-only / mitigation-only / both / neither |
| `compare_funds(fund_ids, metrics?)` | Side-by-side comparison of multiple funds |
| `fund_stage_totals(scope, fund_id?, value?)` | Stage totals for portfolio / single fund / fund_type / sector |

### Finance Tools (5)
| Tool | Description |
|------|-------------|
| `fund_conversion_ratios(fund_name, ...)` | Four pipeline ratios for one fund (deposit/pledge, approval/deposit, disbursement/approval, disbursement/pledge) |
| `rank_entities(filename, group_by, stage, top_k?, ...)` | Rank a grouping column by a finance stage; supports wide and long dataset layouts |
| `rank_by_column(filename, group_by, value_column, top_k?, ascending?, filters?)` | Rank by any numeric column (non-stage columns, ascending order, any dataset) |
| `filter_funds_by_threshold(filename, ratio_metric, threshold, ...)` | Return funds where a ratio metric falls below a threshold |
| `fund_stage_gap(filename?, fund_col?, fund_name?)` | Absolute gaps between consecutive stages (pledge→deposit, deposit→approval, approval→disbursement) |

### Aggregation Tools (1)
| Tool | Description |
|------|-------------|
| `avg_projects_per_fund_by_type(source?, agg?, filters_json?)` | Average approved projects per fund grouped by `fund_type`; supports fund-table fast path or projects-table fine-grained path with filters |

### Quality Tools (1)
| Tool | Description |
|------|-------------|
| `missing_report(filename, group_by, columns, limit?)` | Missing-value counts grouped by a column — audit data completeness before analysis |

---

## Setup

### Prerequisites

- Python 3.12+
- [`uv`](https://docs.astral.sh/uv/) package manager

### Install

```bash
git clone <YOUR_REPO_URL>
cd climate_mcp_server
uv sync
```

Verify the environment:

```bash
uv run python -c "import pandas; print('pandas ok')"
uv run python -c "from mcp.server.fastmcp import FastMCP; print('mcp ok')"
```

Verify the database exists:

```bash
ls "ingestion_layer/data"
# climate_funds.duckdb  manifest.json  archive/  exports/  raw/  raw_snapshots/  ...
```

---

## Running the Server

### Local / STDIO (default)

```bash
uv run python mcp_server/finance/server.py
```

The process stays running and waits for an MCP client connection over STDIO. Stop with `Ctrl+C`.

### HTTP instead of STDIO

The server uses **STDIO** by default. For a VM or remote client, set **`CLIMATE_MCP_TRANSPORT`**:

| Transport | `CLIMATE_MCP_TRANSPORT` | Client protocol |
|-----------|-------------------------|-----------------|
| STDIO (default) | unset or `stdio` | Process stdin/stdout (Claude Desktop, local Cursor) |
| **SSE** | `sse` or `http` | HTTP: GET event stream + POST messages (classic MCP remote) |
| **Streamable HTTP** | `streamable-http` | Single HTTP MCP endpoint (newer transport) |

Optional:

- **`CLIMATE_MCP_HOST`** — bind address (default `0.0.0.0` for HTTP modes so the VM accepts remote connections; `127.0.0.1` for STDIO).
- **`CLIMATE_MCP_PORT`** — listen port (default **8000**).
- **`CLIMATE_MCP_MOUNT_PATH`** — URL prefix (default **`/`**). Non-root mounts apply to every path below.

**SSE example (reachable from your laptop on the lab network):**

```bash
export CLIMATE_MCP_TRANSPORT=sse
export CLIMATE_MCP_HOST=0.0.0.0
export CLIMATE_MCP_PORT=8000
uv run python mcp_server/finance/server.py
```

On startup, stderr prints the exact **GET** (SSE) and **POST** (messages) URLs. Defaults from FastMCP: **`/sse`** and **`/messages/`** under the mount path.

**Streamable HTTP example:**

```bash
export CLIMATE_MCP_TRANSPORT=streamable-http
export CLIMATE_MCP_HOST=0.0.0.0
export CLIMATE_MCP_PORT=8000
uv run python mcp_server/finance/server.py
```

Default MCP path: **`/mcp`** (see stderr line for the full URL).

**Security:** there is **no authentication** on these HTTP endpoints unless you add a reverse proxy (TLS, firewall, API keys). Do not expose them to the public internet without hardening.

You can still use **FastMCP’s** `FASTMCP_*` settings (see library docs) for fine-grained paths like `FASTMCP_SSE_PATH` if needed.

---

## Connecting with Claude Desktop

1. Find your repo root path:
   ```bash
   pwd
   ```

2. In Claude Desktop → Settings → MCP Servers, add a new server:
   - **Name:** `finance`
   - **Transport:** `stdio`
   - **Command:** `uv`
   - **Args:** `run`, `python`, `mcp_server/finance/server.py`
   - **Working directory:** your repo root path

3. Restart Claude Desktop. The 26 tools will appear in the tool picker.

---

## Running Tests

```bash
uv run python tests/run_tests.py --all
```

The test runner requires no external test framework. It runs two layers:

- **Layer 1** — Unit tests for all 26 tools (tool behaviour, error paths, output envelope)
- **Layer 2** — Smoke tests for real-world research questions and use cases (UC1–UC4)

Expected output: `111 PASS | 0 FAIL`

Run individual layers:

```bash
uv run python tests/run_tests.py --layer1   # unit tests only
uv run python tests/run_tests.py --layer2   # smoke tests only
```

---

## Dependencies

| Package | Version | Purpose |
|---------|---------|---------|
| `mcp[cli]` | ≥ 1.26.0 | FastMCP server framework |
| `pandas` | ≥ 3.0.0 | DataFrame processing and aggregations |
| `duckdb` | ≥ 1.4.0 | Primary dataset storage and table queries |
| `httpx` | ≥ 0.28.1 | HTTP transport support |

Python ≥ 3.12 required.