TokLedger
You can query a local TokLedger inference ledger through four read-only MCP tools to inspect usage, costs, and cloud savings.
Get aggregate totals via
tokledger_stats: requests, prompt/completion tokens, avg TTFT, energy (kWh), local cost, cloud-equivalent cost, and percent savings vs cloud.List the most recent ledger records via
tokledger_recent, newest first, with an optional limit of 1–200 (default 20).View a per-model rollup via
tokledger_model_breakdown: requests, tokens, avg TTFT, and local/cloud-equivalent costs.Compare local vs cloud costs via
tokledger_cloud_savings: total local $, total cloud-equivalent $, savings $, and savings %.
Integrates with NVIDIA GPUs via NVML to sample power draw during inference requests, enabling energy (kWh) and local cost calculations when an NVIDIA GPU is available.
Supports Ollama as an upstream local inference engine, forwarding OpenAI-compatible chat completions and using Ollama's usage data for token accounting and cost/energy attribution.
Stores the inference ledger in a local SQLite database, with MCP tools reading aggregate stats, recent records, per-model breakdowns, and cloud savings from that ledger.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@TokLedger how much have I saved vs cloud this week?"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
📈 TokLedger
Quick start · How it works · MCP · Data format · Zero dependencies
The local inference ledger — every token your home GPU serves, priced in tokens, watts, and dollars
💡 The problem this solves
You run a 32B model on your 32 GB rig because API prices hurt. But now nobody knows what inference actually costs you — and the tools that try to answer assume a cloud price list, not your electricity bill.
API observability (Langfuse, Helicone, …) wants your traffic in their cloud.
ollama pstells you VRAM, not cost.The "$/token" number people quote is a cloud number. Your real cost is watts × hours × your tariff — and the comparison that matters is local $ vs cloud $ for the same tokens.
TokLedger is a zero-dependency OpenAI-compatible proxy that sits in front of your local engine and writes one row per request to a local SQLite ledger:
You get | From |
exact or estimated prompt/completion tokens | engine |
TTFT, total latency, tokens/s | measured around the request, streaming-aware |
energy (kWh) + local $ | NVML GPU power sampling when available, else a configurable draw × your |
cloud-equivalent $ | an editable reference price list (gpt-4o-mini by default) |
savings % vs cloud | the number to show people who say "why bother running local" |
Related MCP server: agent-activity
🚀 Quick start
pip install git+https://github.com/adithyanraj03/TokLedger # zero runtime deps
# terminal 1: your engine (llama.cpp shown; Ollama/LM Studio/vLLM work too)
./server -m qwen2.5-32b-instruct-q4_K_M.gguf --port 8080
# terminal 2: the ledger (proxy + dashboard on one port)
tokledger serve --port 8081 --upstream http://127.0.0.1:8080/v1Then point any client at the proxy — the only change is base_url:
from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8081/v1", api_key="none")
r = client.chat.completions.create(model="qwen2.5-32b", messages=[...]) # priced automaticallyOpen http://127.0.0.1:8081/ — the dashboard is live.
No engine handy? There's a deterministic offline mock so the whole pipeline (streaming, usage parsing, costs) runs with zero models:
tokledger mock --port 8999 & # OpenAI-compatible stub
tokledger serve --port 8081 --upstream http://127.0.0.1:8999/v1
curl -s http://127.0.0.1:8081/v1/chat/completions \
-d '{"model":"mock-7b","messages":[{"role":"user","content":"hello"}]}'🖥️ The dashboard
One HTML file, vanilla JS + inline SVG, no CDN, no framework: stats cards, per-day bars with local-vs-cloud cost lines, per-model table, and the recent-requests feed with a per-row est pill whenever token counts are heuristic. Auto-refreshes every 5 s; GET /export.jsonl downloads the full ledger.
⚙️ How it works
Proxy —
POST /v1/chat/completionsis forwarded to your upstream (streaming SSE bytes pass through verbatim).Measure — TTFT is time-to-first-byte for streams; tokens come from the engine's
usagewhen present (Ollama always, llama.cpp/vLLM with stream usage) else from a deterministic ~4-chars/token heuristic, flaggedest.Price —
kWh = watts × seconds / 3.6e6(NVML draw ifpynvml+ NVIDIA GPU present, else configuredload_watts),local $ = kWh × usd_per_kwh,cloud $from the reference price list.Record — one row in
~/.tokledger/ledger.db; config in~/.tokledger/config.json(editable JSON, rate, watts, price list, client tag).
🔌 CLI
tokledger serve --port 8081 --upstream http://127.0.0.1:8080/v1
tokledger dashboard --port 8090 # API + dashboard, no proxy
tokledger mock --port 8999 # offline OpenAI-compatible provider
tokledger stats # totals at a glance
tokledger report -o week.html # static HTML report
tokledger ingest chatlog.jsonl # import (TokLedger or minimal OpenAI usage lines)
tokledger export -o all.jsonl # full ledger out
tokledger mcp # MCP stdio server
tokledger config --usd-per-kwh 0.12 --client swarm-planner🔌 MCP
tokledger mcp speaks MCP over stdio (newline-delimited JSON-RPC, no dependencies) and exposes the ledger to coding agents as four read-only tools — so your agent can answer "what did this week of inference cost?" without you opening a browser:
// mcpServers config (any MCP host)
"tokledger": { "command": "python", "args": ["-m", "tokledger", "mcp"] }Tool | Returns |
| requests, tokens, avg TTFT, kWh, local $, cloud-equiv $, savings % |
| latest ledger records |
| per-model rollup |
| local vs cloud $ + savings |
🧪 Development
pip install pytest
python -m pytest # 50 tests, no GPU, no network beyond localhost
python examples/seed_demo.py # 550 deterministic demo requests for screenshotsThe suite covers token math, cost models, the SQLite ledger, the proxy core with an injected fake upstream (streaming + non-streaming + failures), the mock provider over real sockets, the full end-to-end proxy→ledger path, ingest/export, the MCP protocol in-process and over a subprocess, and CLI smoke tests.
🤔 Honest scope
Token counts are exact when your engine reports
usage(recommended: enable it); otherwise they're a clearly-labelled heuristic estimate.Energy attribution assumes the GPU draw during the request window. Multi-GPU and CPU-side energy are out of scope; NVML covers NVIDIA (AMD/Intel fall back to the configured draw).
The cloud price list is a reference, not a quote — edit
cloud_pricesinconfig.jsonfor your region/provider.Single process, localhost-first: it's not a multi-tenant gateway; bind it to
127.0.0.1(the default) and it never needs the internet.
🔒 Privacy
No telemetry, no accounts, no cloud, no CDN. One SQLite file, one JSON config, one process on 127.0.0.1. The mock provider and the full test suite run entirely offline.
📄 License
MIT — see LICENSE.
📬 Contact
Adithya N Raj · GitHub · adithyanraj03@gmail.com · LinkedIn
© 2026 Adithya N Raj ✨
Available Tools
4 toolstokledger_cloud_savingsB
Local vs cloud cost comparison: total local $, total cloud-equivalent $, savings $ and %.
Inputs: no named parameters.
tokledger_model_breakdownB
Per-model rollup: requests, tokens, avg TTFT, local and cloud-equivalent cost.
Inputs: no named parameters.
tokledger_recentB
The most recent N ledger records (newest first).
Inputs: limit.
tokledger_statsB
Aggregate totals from the local LLM inference ledger: request count, prompt/completion tokens, average TTFT, energy (kWh), local cost ($), cloud-equivalent cost ($), and percent saved vs cloud.
Inputs: no named parameters.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 4 tools
Tools are mostly distinct: stats for aggregate metrics, recent for raw records, model_breakdown for per-model rollup, and cloud_savings for cost comparison. However, tokledger_stats already includes cloud-equivalent cost and percent saved, which overlaps with tokledger_cloud_savings, creating minor ambiguity for cost-focused queries.
All tool names share the 'tokledger_' prefix and use consistent snake_case noun phrases. While not verb_noun, the convention is uniform and predictable.
Four tools is well-scoped for a ledger analytics server, covering summary, recent records, per-model breakdown, and savings. Each tool has a clear role without redundancy.
Core analytical queries are covered, but the surface lacks filtering by date range or model, and no pagination for recent records. These are minor gaps that agents could work around by adjusting N or using aggregate stats.
Maintenance
Related MCP Connectors
Analytics for MCP servers. Query your tool calls, first-call success, retries and schema cost.
Ledger of record for AI spend: verifiable receipts, waste auditor & model arbitrage MCP.
Veqen brings together work and coding agent costs per person and team, product inference costs per customer, agent spending, protection incidents, invoice checks, chargeback and the month-end close. Ask your assistant to investigate recorded spending, compare model prices, review provider keys and incidents, record bills and credits, or manage budgets, alerts, teams and spending policies. Your account role and plan govern access. Provider-key shut-off and agent-card controls require the appropriate role and provider permissions. Destructive actions require confirmation, and organizations can require a second person to approve a shut-off. SDK budget blocking is not a guaranteed spending cap. Work and coding agents are free for up to 3 people; product inference is free up to $1,000 in monthly AI spend. Connect your Veqen account for private organization data. The separate public model-price server needs no account. Estimates remain separate from provider invoices. Setup and permissions: https://veqen-app.com/docs/mcp
Veqen brings together work and coding agent costs per person and team, product inference costs per customer, agent spending, protection incidents, invoice checks, chargeback and the month-end close. Ask your assistant to investigate recorded spending, compare model prices, review provider keys and incidents, record bills and credits, or manage budgets, alerts, teams and spending policies. Your account role and plan govern access. Provider-key shut-off and agent-card controls require the appropriate role and provider permissions. Destructive actions require confirmation, and organizations can require a second person to approve a shut-off. SDK budget blocking is not a guaranteed spending cap. Work and coding agents are free for up to 3 people; product inference is free up to $1,000 in monthly AI spend. Connect your Veqen account for private organization data. The separate public model-price server needs no account. Estimates remain separate from provider invoices. Setup and permissions: https://veqen-app.com/docs/mcp
Related MCP Servers
- AlicenseAqualityBmaintenanceA local-first, multi-provider cost meter for LLM usage, exposed as MCP tools. Captures every call into a local SQLite ledger and lets any coding agent query spend, compare providers, and get recommendations — no cloud, no account. First-class support for Chinese providers (Qwen, DeepSeek) alongside Anthropic and OpenAI.790 PyPI6MIT
- AlicenseAqualityBmaintenanceA read-only MCP server that exposes local coding-agent session logs as three tools for introspection of recent work, debugging tool failures, and tracking token usage and estimated cost without parsing log files.3MIT
- AlicenseNot gradedqualityDmaintenanceProvides financial audit reporting and ledger query tools for AI agents, enabling generation of compliance, risk, and transaction reports and querying of transaction data, balances, and summaries from an in-memory SQLite database via MCP.MIT
- AlicenseNot gradedqualityBmaintenanceRead-only MCP server that lets local AI agents query bank transactions stored in a local SQLite database, with tools for accounts, transaction search, spending summaries, recurring charges, trends, themes, subscriptions, and store health.2MIT