llm-cost-mcp

A small, read-only MCP server that reports LLM cost, latency and failures per model. It is a portfolio demo built on synthetic data, and a pattern you can adapt for any multi-model LLM application.

Demo notice. Every row comes from make_sample_data.py. The model names (alpha-large, beta-pro, ...) are fictional, the prices are illustrative, and nothing comes from a real system. The improvement visible in the second half of the data (fewer calls, shorter outputs, fewer timeouts) is written into the generator on purpose; it is not a finding. This project makes no claims about users, production use or real savings.

What it does

An AI client connected to this server can ask:

• Where does the money go? (cost_by_model)

• How did this period compare with the one before? (compare_periods)

• How much of the change came from fewer calls and how much from cheaper calls? (savings_summary)

• Which provider fails most often, and how many failures did a fallback rescue? (failure_rate_by_provider)

• Which individual calls were slowest? (slowest_analyses)

It also offers the table description as a resource (schema://analyses) and a ready-made prompt (monthly_cost_review) that walks an AI through a monthly review.

Related MCP server: llmkit-mcp-server

Quick start

Requires Python 3.10+ (tested on Python 3.13 with mcp 1.30.0, on Linux).

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements-dev.txt
python make_sample_data.py      # writes sample.db (synthetic)
python -m pytest -q             # run the tests
python server.py                # MCP server over stdio; waits for a client

The server is pinned to mcp<2: version 2 of the Python SDK renamed FastMCP, and this demo uses the 1.x API.

To use it from an MCP client, register the server with absolute paths. A typical client configuration looks like this (not verified here against a specific desktop client; the automated tests use the official Python client over stdio, see tests/test_stdio_client.py):

{
  "mcpServers": {
    "llm-cost-demo": {
      "command": "/absolute/path/to/.venv/bin/python",
      "args": ["/absolute/path/to/server.py"]
    }
  }
}

Environment variables (both optional): LLM_COST_DB (database path, default ./sample.db) and LLM_COST_AUDIT (audit log path, default ./audit.jsonl).

Tools

Every docstring is written for an AI reader: what it returns, units and limits. Costs are USD from an illustrative price table.

Resource schema://analyses: column names, types, units and meanings, row count and date range. It contains no row-level data, and a column without a description is flagged as undocumented column rather than hidden.

Prompt monthly_cost_review(period_days=30): instructions for an AI to read the schema, call the tools in a sensible order and write a review. It tells the AI to quote only returned numbers, keep observation and interpretation apart, not to forecast, and not to claim real-world savings.

Why read-only, and how

1. The SQLite connection is opened with mode=ro and PRAGMA query_only = ON. Tests prove that INSERT, UPDATE, DELETE, DROP and CREATE are refused, and refused even when query_only is switched off again.

2. No tool accepts SQL. Every query is fixed in the code and parameterised.

3. Inputs are validated with bounds: integers only (booleans and strings are rejected), within the ranges above.

4. There are no write tools.

5. The one file the server writes is the audit log, which is separate from the data.

Audit log

Every tool call appends one JSON line to audit.jsonl:

{"ts":"2026-11-06T13:33:05.147Z","tool":"cost_by_model","args":{"period_days":7},"status":"ok","error_type":null,"row_count":5,"duration_ms":1.91}

• status is ok, error (the tool ran and raised) or rejected (the call never reached the tool: schema validation failed or the tool name is unknown).

• Result contents are never logged, only a row count. Error messages are never logged, only the exception class name.

• Arguments are sanitised: strings are cut to 50 characters with control characters removed, other types are replaced by their type name, and rejected calls keep at most 10 arguments.

• The file is created owner-only (0600). If writing fails, the tool still works and a short note goes to stderr.

Design notes

• Windows are measured back from the newest row, not from the clock, so results are reproducible for a static database. A window excludes its start and includes its end.

• Failures are attributed to the provider tried first. A fallback row is a call whose first attempt failed and was served by another provider; the failure counts against the first provider, and the rescuing provider sees it as served_as_fallback.

• Failed calls cost 0 in this demo (assumed not billed) but still count as calls.

• compare_periods and savings_summary report how much of the earlier period the data actually covers (previous_coverage_pct).

Limits and caveats

• This is a demo, not a hardened service. It is meant for local stdio use: there is no authentication, rate limiting or multi-user separation.

• The audit log is a plain local file. It is not tamper-evident, and resource reads and prompt requests are not logged (only tool calls and rejected tool calls are).

• The protocol layer (FastMCP) ignores unknown tool arguments and converts numeric strings such as "7" to integers before this server's validation runs.

• Tested on one platform and one SDK version (see Quick start). There are no performance claims; the sample has about 2,000 rows.

• All figures in the sample are synthetic.

Using the pattern with your own application

1. Export your usage into a copy (a SQLite file) with the same columns as schema://analyses. Do not point this demo at a live production database.

2. Keep the columns categorical or numeric. Do not store prompts, completions, user identifiers or other free text in them.

3. Set LLM_COST_DB to your file and read the schema resource to check the columns.

4. Review the privacy and security of your own data before exposing any tool to an AI client; this demo has not been reviewed for that.

Project layout

server.py               tools, resource and prompt
db.py                   read-only connection, input validation, period maths
audit.py                audit log and the protocol-level hook for rejected calls
make_sample_data.py     generates the synthetic sample.db
tests/                  pytest suite, including an end-to-end test with the MCP Python client
AGENTS.md               instructions for an AI agent using the server

Tests

python -m pytest -q

The suite covers the tool maths on a small hand-built database, input validation, the read-only guarantees, the audit log (including that results are never written), and a real client talking to the server over stdio. A test also checks that these docs mention every tool and its arguments.

License

MIT (see LICENSE).