Inferrail
Read-only MCP server for querying Inferrail spend and gateway health without running inference or incurring provider costs.
get_spend: aggregates the local receipt ledger by provider, model, route, or custom business attribute such ascustomeroragent, optionally within a time window.get_spend: returns known USD cost as decimal strings plus a count of requests whose pricing was unresolvable, avoiding silently treating them as zero.get_spend: reads a local receipts file only; it makes no network call and cannot incur provider cost.get_health: checks whether an Inferrail gateway is reachable via its/healthendpoint atbase_urland reports the most recent local receipt, if any.get_health: does not make an inference call or spend provider budget.Both tools are read-only and accept an optional
receipts_pathto locate local receipt data.
Tracks token usage and estimated LLM cost for OpenAI-compatible chat completion endpoints, including streaming and tool calls, by routing requests through the Inferrail gateway.
What works today
Developer preview: interfaces may change before 1.0.
Capability | What it does | Where | Status |
Gateway | OpenAI- and Anthropic-compatible proxy you run yourself. A dollar budget per job, shared by parallel calls; a call that would exceed it gets HTTP 402 before the provider. |
| Available |
Cost receipts | One record per call: tokens, price, cost, status, your tags. Never prompts or responses. Totals by job, customer or any tag. | local SQLite or JSONL | Available |
MCP server | Read-only |
| Preview |
Inferrail Hosted | The same per-run budgets run for you, shared across agents and machines. 250 runs a month free, then USDC credits over x402. Bring your own provider key. | Preview | |
Web Read | One public web page as clean markdown with a receipt, paid per call by the agent over x402 or MPP. | Preview | |
Agent purchases under a job budget | One budget covering model calls and x402 purchases; a purchase that doesn't fit is refused before it's signed. Local proof, no mainnet. | Experimental |
Also experimental: AP invoice-exception recovery, a hosted trial gateway, and Work Economics / Economic Authority (example; Base Sepolia testnet only).
Related MCP server: AgentCost MCP Server
Quickstart
Requires Python 3.11+.
Try it offline. No API key, no network calls, no provider charges:
pip install inferrail
inferrail demo
inferrail report --by customer --receipts ./inferrail-demo-receipts.jsonlThe demo sends scripted requests through the real engine to a fake provider with made-up
prices labelled DEMO, and prints what they cost by customer.
Give one run a budget, inside your Python app. No config file, no second process:
import inferrail
from openai import OpenAI
base_url = inferrail.start() # the gateway, on a background thread in this process
client = OpenAI(base_url=base_url, api_key="unused") # your key stays in OPENAI_API_KEY
client.chat.completions.create(
model="gpt-4o-mini", # any model your account can use; `inferrail models` lists priced ones
max_tokens=200,
messages=[{"role": "user", "content": "Summarize this contract."}],
extra_headers={
"X-Inferrail-Attribute-Work-Id": "contract-review-42", # the run
"X-Inferrail-Budget-Usd": "0.50", # its dollar ceiling
},
)Every call with the same work id shares that budget, including parallel calls; one that would
push the run past it gets HTTP 402 before it reaches OpenAI. inferrail work contract-review-42
shows what the run cost.
More: run it as a shared gateway with the local dashboard, framework snippets (LangChain, LangGraph, OpenAI Agents SDK, CrewAI, Haystack, LlamaIndex, Microsoft Agent Framework), and other clients, including the Anthropic SDK.
For agents
Start at tryinferrail.com/llms.txt: what's live, with links to each service's guide and the OpenAPI contract.
Web Read sells one public web page as clean markdown, with its word count, a SHA-256 of the
content and a receipt. $0.002 per delivered page, paid over x402 (USDC
on Base) or MPP (Tempo); no account or API key. The payment is checked first
and settled only after the page is delivered, so refusals (robots.txt, non-public addresses,
non-text, over 2 MB, target-site errors) are never charged. Retrying the same url and
request_id from the same wallet within 10 minutes returns the page already paid for.
curl -i -X POST https://inferrail-cost-gateway.onrender.com/read/v1/page \
-H 'content-type: application/json' -d '{"url": "https://example.com/"}'
# HTTP 402 with the price; an x402 or MPP client pays and repeats the requestGuides: Web Read · Hosted · service source.
MCP. Ask your agent what a job cost, from your local receipts:
claude mcp add inferrail \
-e INFERRAIL_RECEIPTS_PATH=/absolute/path/to/inferrail-receipts.jsonl \
-- uvx --with "mcp>=2.0" inferrail mcpTwo read-only tools, get_spend (known cost and tokens by provider, model, route or any tag
such as work_id) and get_health. They don't run inference, write files or pay for anything.
Other clients: docs/integrations.md.
How it works
The gateway routes each request by model to a configured provider, checks any budget in
scope, forwards the call, and writes one receipt per request, including requests a budget
refused. A cost is recorded only when the provider reports usage and a price is on file;
otherwise it's unknown, never a guessed $0.
Where | What it sees or keeps |
Your provider | The full request, exactly as it would without Inferrail, under the provider's own policies. |
The gateway, in memory | Your provider key (from its own environment) and the prompts and responses it forwards. |
Receipts, on your disk | Provider, model, token usage, the price used and the cost, status, timing, and the attribution tags you send. |
Not in receipts | Prompt and response bodies. |
Attribution tags are stored as sent, so keep secrets and message content out of them.
inferrail verify-payload-free prints the live receipt schema, and canary tests check that
message bodies never reach receipts; neither is a security audit. The optional usage beacon
sends nothing unless an endpoint is configured (details).
Architecture: docs/ARCHITECTURE.md.
Current support
Endpoints:
POST /v1/chat/completions(OpenAI-compatible) andPOST /v1/messages(Anthropic-compatible), with streaming and tool calls, plusGET /v1/models.Providers: OpenAI, Anthropic, and endpoints compatible with either. Built-in prices cover OpenAI and Anthropic models; others need a price in your config, or their cost stays unknown.
Not supported: the OpenAI Responses API, embeddings, image generation, audio and Realtime, batch, and native Gemini or Bedrock APIs. Fields the gateway can't account for are rejected with a clear error, never silently dropped.
Scope: only calls through a running gateway are counted.
Exact contract and non-goals: docs/PRODUCT.md. Running agent jobs that spend real money? Open a production setup request.
Documentation
Integrations: SDKs, frameworks, attribution, MCP, voice
Self-hosting: install, configuration, storage, budgets, dashboard
Machine-readable: llms.txt, openapi.json (the gateway), config.schema.json
Contributing, security, license
Questions and bugs: GitHub issues.
Security or privacy vulnerabilities: report privately, as described in
SECURITY.md. Contributing: CONTRIBUTING.md
(pytest needs no API key or network). License: Apache-2.0.
Available Tools
2 toolsget_healthCheck gateway health and recent attributionA
Check whether an Inferrail gateway is reachable at base_url via its /health endpoint, and separately report the most recent local receipt (if any) as evidence of recent attributed activity. Does NOT make a new inference call -- a health check must never silently spend provider budget as a side effect.
Inputs: base_url, receipts_path. Returns: object
get_spendQuery attributed spendA
Query Inferrail's local receipt ledger, aggregated by a dimension (provider, model, route, or any business attribute name such as 'customer' or 'agent'), optionally restricted to a time window. Returns known cost in USD (as a decimal string, never a float) per group, plus a count of requests whose pricing was unresolvable -- those are never silently folded into the cost total as zero. Reads a local file only; makes no network call and cannot incur provider cost.
Inputs: by, since, until, receipts_path. Returns: object
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 2 tools
get_health and get_spend are clearly distinct: one checks gateway reachability and recent receipt, the other aggregates spend from the local ledger. There is no overlap in purpose or side effects.
Both tools follow the same get_ verb_noun convention (get_health, get_spend), creating a predictable and consistent pattern for the server's small surface.
With only 2 tools, the server feels thin, though the narrow scope of health and spend monitoring makes the count understandable. It is at the low end of acceptable for a focused read-only utility set.
The server covers the two core monitoring concerns for an Inferrail gateway: health and spend. Minor gaps like a tool to list available routes or models exist, but the current surface is sufficient for basic observability.
Maintenance
Related MCP Connectors
Cost per run, tool, model and day for AI agents, from one call or an OpenTelemetry export.
AI spend by team, developer and API key for Claude, OpenAI, Gemini and coding agents.
1Enforce AI budgets before the model call and track cost per customer across 10 providers.
Budget & cost control for AI agents — per-agent spend caps + rate limits before each call.
Related MCP Servers
AlicenseNot gradedqualityDmaintenanceProvides cost intelligence and a reputation scoring system to help AI agents optimize spending through smart model selection and local-to-cloud routing. It enables real-time cost tracking and rewards agents for making efficient, high-credibility decisions across various LLM providers.Apache 2.0- AlicenseNot gradedqualityDmaintenanceProvides real-time AI model pricing, cost estimation, and budget management tools to help agents understand and optimize their spending. It enables agents to compare costs across multiple providers and select the most cost-effective models for specific tasks.1MIT
- AlicenseAqualityDmaintenanceTracks AI agent token usage and spending in real time, with budget alerts, per-task cost breakdown, and a visual dashboard.61MIT
- FlicenseNot gradedqualityDmaintenanceEnables AI agents to track LLM costs, enforce budgets, compare models, and estimate expenses through simple tool calls.-