Skip to main content
Glama

What works today

Developer preview: interfaces may change before 1.0.

Capability

What it does

Where

Status

Gateway

OpenAI- and Anthropic-compatible proxy you run yourself. A dollar budget per job, shared by parallel calls; a call that would exceed it gets HTTP 402 before the provider.

pip install inferrail

Available

Cost receipts

One record per call: tokens, price, cost, status, your tags. Never prompts or responses. Totals by job, customer or any tag.

local SQLite or JSONL

Available

MCP server

Read-only get_spend and get_health tools over your local receipts.

inferrail mcp

Preview

Inferrail Hosted

The same per-run budgets run for you, shared across agents and machines. 250 runs a month free, then USDC credits over x402. Bring your own provider key.

hosted guide

Preview

Web Read

One public web page as clean markdown with a receipt, paid per call by the agent over x402 or MPP.

below

Preview

Agent purchases under a job budget

One budget covering model calls and x402 purchases; a purchase that doesn't fit is refused before it's signed. Local proof, no mainnet.

example

Experimental

Also experimental: AP invoice-exception recovery, a hosted trial gateway, and Work Economics / Economic Authority (example; Base Sepolia testnet only).

Related MCP server: AgentCost MCP Server

Quickstart

Requires Python 3.11+.

Try it offline. No API key, no network calls, no provider charges:

pip install inferrail
inferrail demo
inferrail report --by customer --receipts ./inferrail-demo-receipts.jsonl

The demo sends scripted requests through the real engine to a fake provider with made-up prices labelled DEMO, and prints what they cost by customer.

Give one run a budget, inside your Python app. No config file, no second process:

import inferrail
from openai import OpenAI

base_url = inferrail.start()   # the gateway, on a background thread in this process
client = OpenAI(base_url=base_url, api_key="unused")   # your key stays in OPENAI_API_KEY
client.chat.completions.create(
    model="gpt-4o-mini",   # any model your account can use; `inferrail models` lists priced ones
    max_tokens=200,
    messages=[{"role": "user", "content": "Summarize this contract."}],
    extra_headers={
        "X-Inferrail-Attribute-Work-Id": "contract-review-42",   # the run
        "X-Inferrail-Budget-Usd": "0.50",                        # its dollar ceiling
    },
)

Every call with the same work id shares that budget, including parallel calls; one that would push the run past it gets HTTP 402 before it reaches OpenAI. inferrail work contract-review-42 shows what the run cost.

More: run it as a shared gateway with the local dashboard, framework snippets (LangChain, LangGraph, OpenAI Agents SDK, CrewAI, Haystack, LlamaIndex, Microsoft Agent Framework), and other clients, including the Anthropic SDK.

For agents

Start at tryinferrail.com/llms.txt: what's live, with links to each service's guide and the OpenAPI contract.

Web Read sells one public web page as clean markdown, with its word count, a SHA-256 of the content and a receipt. $0.002 per delivered page, paid over x402 (USDC on Base) or MPP (Tempo); no account or API key. The payment is checked first and settled only after the page is delivered, so refusals (robots.txt, non-public addresses, non-text, over 2 MB, target-site errors) are never charged. Retrying the same url and request_id from the same wallet within 10 minutes returns the page already paid for.

curl -i -X POST https://inferrail-cost-gateway.onrender.com/read/v1/page \
  -H 'content-type: application/json' -d '{"url": "https://example.com/"}'
# HTTP 402 with the price; an x402 or MPP client pays and repeats the request

Guides: Web Read · Hosted · service source.

MCP. Ask your agent what a job cost, from your local receipts:

claude mcp add inferrail \
  -e INFERRAIL_RECEIPTS_PATH=/absolute/path/to/inferrail-receipts.jsonl \
  -- uvx --with "mcp>=2.0" inferrail mcp

Two read-only tools, get_spend (known cost and tokens by provider, model, route or any tag such as work_id) and get_health. They don't run inference, write files or pay for anything. Other clients: docs/integrations.md.

How it works

The gateway routes each request by model to a configured provider, checks any budget in scope, forwards the call, and writes one receipt per request, including requests a budget refused. A cost is recorded only when the provider reports usage and a price is on file; otherwise it's unknown, never a guessed $0.

Where

What it sees or keeps

Your provider

The full request, exactly as it would without Inferrail, under the provider's own policies.

The gateway, in memory

Your provider key (from its own environment) and the prompts and responses it forwards.

Receipts, on your disk

Provider, model, token usage, the price used and the cost, status, timing, and the attribution tags you send.

Not in receipts

Prompt and response bodies.

Attribution tags are stored as sent, so keep secrets and message content out of them. inferrail verify-payload-free prints the live receipt schema, and canary tests check that message bodies never reach receipts; neither is a security audit. The optional usage beacon sends nothing unless an endpoint is configured (details). Architecture: docs/ARCHITECTURE.md.

Current support

  • Endpoints: POST /v1/chat/completions (OpenAI-compatible) and POST /v1/messages (Anthropic-compatible), with streaming and tool calls, plus GET /v1/models.

  • Providers: OpenAI, Anthropic, and endpoints compatible with either. Built-in prices cover OpenAI and Anthropic models; others need a price in your config, or their cost stays unknown.

  • Not supported: the OpenAI Responses API, embeddings, image generation, audio and Realtime, batch, and native Gemini or Bedrock APIs. Fields the gateway can't account for are rejected with a clear error, never silently dropped.

  • Scope: only calls through a running gateway are counted.

Exact contract and non-goals: docs/PRODUCT.md. Running agent jobs that spend real money? Open a production setup request.

Documentation

Contributing, security, license

Questions and bugs: GitHub issues. Security or privacy vulnerabilities: report privately, as described in SECURITY.md. Contributing: CONTRIBUTING.md (pytest needs no API key or network). License: Apache-2.0.

Available Tools

2 tools
get_healthCheck gateway health and recent attributionA

Check whether an Inferrail gateway is reachable at base_url via its /health endpoint, and separately report the most recent local receipt (if any) as evidence of recent attributed activity. Does NOT make a new inference call -- a health check must never silently spend provider budget as a side effect.

Inputs: base_url, receipts_path. Returns: object

get_spendQuery attributed spendA

Query Inferrail's local receipt ledger, aggregated by a dimension (provider, model, route, or any business attribute name such as 'customer' or 'agent'), optionally restricted to a time window. Returns known cost in USD (as a decimal string, never a float) per group, plus a count of requests whose pricing was unresolvable -- those are never silently folded into the cost total as zero. Reads a local file only; makes no network call and cannot incur provider cost.

Inputs: by, since, until, receipts_path. Returns: object

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A4.1/5.0

    Scored across 2 tools

    Disambiguation5/5

    get_health and get_spend are clearly distinct: one checks gateway reachability and recent receipt, the other aggregates spend from the local ledger. There is no overlap in purpose or side effects.

    Naming Consistency5/5

    Both tools follow the same get_ verb_noun convention (get_health, get_spend), creating a predictable and consistent pattern for the server's small surface.

    Tool Count3/5

    With only 2 tools, the server feels thin, though the narrow scope of health and spend monitoring makes the count understandable. It is at the low end of acceptable for a focused read-only utility set.

    Completeness4/5

    The server covers the two core monitoring concerns for an Inferrail gateway: health and spend. Minor gaps like a tool to list available routes or models exist, but the current surface is sufficient for basic observability.

    Maintenance

    ActivityActive
    ResponsivenessLimited data

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      D
      maintenance
      Provides cost intelligence and a reputation scoring system to help AI agents optimize spending through smart model selection and local-to-cloud routing. It enables real-time cost tracking and rewards agents for making efficient, high-credibility decisions across various LLM providers.
      Apache 2.0
    • A
      license
      Not graded
      quality
      D
      maintenance
      Provides real-time AI model pricing, cost estimation, and budget management tools to help agents understand and optimize their spending. It enables agents to compare costs across multiple providers and select the most cost-effective models for specific tasks.
      1
      MIT