runmeter
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@runmetershow me cost summary for last 7 days grouped by model"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
runmeter
An MCP server that gives any LLM or agent workflow cost and reliability observability for free.
Record one row per model call (model, tokens, cost, latency, finish reason, tags), then query and aggregate that telemetry through MCP tools. Cost is computed automatically from a built-in, overridable pricing table when you do not pass an explicit figure. Storage is a local SQLite file. No external service, no credentials, no network calls.
Built with FastMCP. Works with any MCP client (Claude Code, Claude Desktop, or your own agent) over stdio.
Companion project: a-agentic-delivery-pipeline is an AI-native, skills-driven delivery pipeline that emits its per-stage telemetry to runmeter.
Why
Most agent stacks can tell you what an agent did, but not what it cost or how often it failed. Teams end up bolting on a spreadsheet or a bespoke logging table per project. runmeter makes per-run cost and reliability a first-class, queryable signal that every agent on the machine writes to the same way. It is the "cost-per-task economics" and "evaluation and performance" discipline that production agent work needs, packaged as a drop-in MCP server.
Related MCP server: ai-usage-metrics-mcp
Tools
Tool | Purpose | Hint |
| Record one model/agent run; auto-computes cost when the model is priced | write |
| Fetch a single run by id | read-only |
| List runs newest-first with filters (model, agent, tag, status, since) and paging | read-only |
| Roll up cost, tokens, avg latency, and error rate grouped by model / agent / finish_reason / day / tag | read-only |
| Export raw runs as JSON or CSV for external analysis | read-only |
| Delete one run by id; requires | destructive |
Every tool returns both a human-readable note and structured content.
Install
git clone https://github.com/Aptica-Solutions/a-mcp-runmeter.git
cd a-mcp-runmeter
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txtRun
python3 server.pyConnect from an MCP client
Point your client at server.py over stdio. Example (claude_mcp_config.json):
{
"mcpServers": {
"runmeter": {
"command": "python3",
"args": ["/absolute/path/to/a-mcp-runmeter/server.py"],
"env": { "RUNMETER_DB_PATH": "~/.runmeter/runmeter.db" }
}
}
}Configuration
All optional. runmeter runs with zero configuration.
Variable | Default | Purpose |
|
| SQLite telemetry store location |
| built-in table | JSON file overriding/extending per-model pricing |
Pricing override file shape (USD per 1,000,000 tokens):
{
"my-fine-tuned-model": { "input": 3.0, "output": 15.0 }
}The built-in table ships sensible list-price defaults for common Claude, GPT, and Gemini models. They are convenience defaults, not a billing source of truth; override them for anything you meter seriously.
Example flow
An agent records each call as it goes:
// runmeter_record
{ "run": { "model": "claude-sonnet-4", "input_tokens": 8200, "output_tokens": 640,
"latency_ms": 1830, "finish_reason": "stop", "agent": "discharge-summarizer",
"tags": ["prod", "east"] } }
// -> cost auto-computed at $3/M in + $15/M out = $0.034200Then you ask for a rollup:
// runmeter_summary { "group_by": "model", "since": "7d" }
// -> per-model run count, tokens, total cost, avg latency, and error rate,
// sorted by cost descending, with grand totals.since accepts an ISO timestamp or a relative window: 30m, 24h, 7d, 2w.
Development
pip install -e ".[dev]"
pytest -qThe test suite points RUNMETER_DB_PATH at a throwaway file per test, so it never touches a real store.
Design notes
Tools use consistent
runmeter_prefixes for discoverability.Read tools carry
readOnlyHint; the single delete tool carriesdestructiveHintand refuses to run without an explicitconfirm, so it cannot wipe the store by accident.Aggregation runs in Python so multi-tag rows and per-day buckets are handled uniformly, keeping the SQL simple and portable.
No ORM and one dependency beyond the MCP SDK and Pydantic, by design: low total cost of ownership and easy to audit.
License
MIT. See LICENSE.
Built by Aptica Solutions.
Available Tools
6 toolsrunmeter_deleteADestructiveIdempotent
Delete a single run by id. Requires confirm=true to actually remove it.
This is the only destructive tool. It scopes to one id and refuses to run without an explicit confirm, so it cannot wipe the store by accident.
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | ||
| confirm | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Despite annotations already declaring destructiveHint=true, the description adds crucial behavioral context: the requirement for confirm=true, scoping to a single ID, and the safety guarantee that it cannot wipe the store accidentally. This goes beyond the structured annotations and is highly valuable for an agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the primary action, and every word adds value. It is concise without omitting essential safety details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the tool's purpose, safety guardrail, and uniqueness among siblings. With an output schema present, it doesn't need to explain return values. It lacks only minor details like what happens if the run_id doesn't exist, but this is not critical given the overall completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description mentions both parameters indirectly: 'by id' refers to run_id, and 'requires confirm=true' explains the confirm parameter's significance. With 0% schema coverage, this textual explanation compensates well, though it doesn't spell out the boolean type or default value which are in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Delete a single run by id,' which is a specific verb+resource+scope. It clearly distinguishes from sibling tools by noting 'This is the only destructive tool,' making its purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states that this is the only destructive tool and requires explicit confirm=true, implying it should be used for one-off deletions with caution. It doesn't explicitly name alternative tools for retrieval or other operations, but the context is clear enough to guide an agent on when to invoke it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
runmeter_exportARead-onlyIdempotent
Export raw runs as a JSON array or CSV string for external analysis.
fmt is 'json' or 'csv'. Optional since limits the window. Newest-first,
capped at limit rows (max 10000).
| Name | Required | Description | Default |
|---|---|---|---|
| fmt | No | json | |
| limit | No | ||
| since | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the readOnly/idempotent annotations, the description adds important behavioral details: output is JSON or CSV, ordered newest-first, capped at limit rows, with a configurable since window and max limit of 10000. These details help the agent predict exactly what happens without contradicting the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loads the main purpose, and every sentence adds useful information. It avoids repetition of schema defaults and stays compact.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read-only export tool with three parameters and an output schema, the description is nearly complete. It covers purpose, output format, ordering, and limits. Minor omissions are the exact format of 'since' and lack of explicit guidance vs sibling tools, but these do not critically impair usability.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, so the description carries the burden. It does explain fmt values, since as a time window, and limit as a row cap with a maximum. The main gap is that the exact format of 'since' (e.g., ISO date, timestamp) is not specified, leaving some ambiguity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Export'), a clear resource ('raw runs'), and output formats ('JSON array or CSV string'), which clearly distinguishes this from siblings like runmeter_get or runmeter_list. It focuses on bulk export rather than retrieval, summary, or mutation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'for external analysis' provides a clear context for when to use this tool, and the behavior is described concretely. However, it does not explicitly mention alternatives or when NOT to use it relative to sibling tools, so it falls short of perfect guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
runmeter_getARead-onlyIdempotent
Return the full stored row for a single run id.
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true. The description adds the 'full stored row' detail, which provides some value beyond the schema, but it does not mention behavior for nonexistent IDs or permission requirements. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with zero filler. It front-loads the action immediately and wastes no words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple, and the presence of an output schema covers return structure. Annotations cover the read-only and idempotent nature. For a basic get-by-ID operation, the description is sufficiently complete, though it omits error-case behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With only one parameter (run_id) and 0% schema description coverage, the description is expected to compensate. It mentions 'run id', which aligns with the schema title, but adds no additional semantics beyond what the parameter name itself implies. Adequate but not enhancing.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Return' and clearly identifies the resource ('full stored row') and scope ('single run id'). This distinguishes it from sibling tools like list (multiple runs) and summary (aggregate data).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when fetching a single run's full data, but it does not explicitly state when to use this tool over alternatives like summary or list. No exclusions or alternative tool names are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
runmeter_listARead-onlyIdempotent
List runs newest-first with optional filters.
Filters: model, agent, status ('ok'/'error'), tag (matches any run carrying
that tag), and since (ISO timestamp or relative window like '24h', '7d').
Use limit/offset to page. Returns up to limit rows plus the total count
matching the filters so you know whether to page further.
| Name | Required | Description | Default |
|---|---|---|---|
| tag | No | ||
| agent | No | ||
| limit | No | ||
| model | No | ||
| since | No | ||
| offset | No | ||
| status | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations (readOnlyHint, idempotentHint), the description adds meaningful behavioral detail: newest-first ordering, filter semantics (including status allowed values 'ok'/'error' and tag match behavior), relative time windows for 'since', and the return of total count to signal pagination needs. This goes beyond basic safety hints and enriches the agent's understanding of what to expect.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and well-structured: a one-line summary sentence, a concise filter list with explanations, and a final sentence on pagination and return count. Every sentence earns its place with no redundant or vague wording.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a list tool with 7 optional parameters, an output schema, and strong annotations, this description is complete. It covers purpose, ordering, all filter options with semantics, pagination behavior, and the return shape (limited rows plus total count). No critical gaps are evident.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description fully compensates by explicitly listing all seven parameters (tag, agent, limit, model, since, offset, status) and explaining their semantics. It clarifies status allowed values, since format (ISO or relative like '24h'/'7d'), tag matching, and pagination use of limit/offset. This adds substantial meaning beyond the bare schema field names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'List runs newest-first with optional filters.' This uses a specific verb ('list') plus resource ('runs') and adds ordering and filtering scope. It distinguishes from sibling tools like runmeter_get (single run fetch), runmeter_summary (aggregates), runmeter_record (create), and runmeter_delete (delete) by emphasizing the paginated list behavior with filters.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear usage context: it explains the available filters and how to use limit/offset for pagination, including how to know when to page further via the total count. However, it does not explicitly name sibling alternatives or state when not to use this tool (e.g., for a single run use runmeter_get). This matches 'clear context, no exclusions'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
runmeter_recordA
Record a single LLM/agent run and return the stored row.
Cost is taken from cost_usd when provided; otherwise it is computed from
the built-in pricing table (USD per 1M tokens) when the model is known. If
the model is unpriced and no cost is given, cost is stored as null and the
response notes that the model is unpriced so you can extend the table.
| Name | Required | Description | Default |
|---|---|---|---|
| run | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds substantial value beyond the basic annotations by detailing cost-handling logic: explicit cost_usd takes precedence, otherwise computed from a pricing table, and unpriced models result in null cost with a response note. This helps the agent predict behavior without contradicting the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short paragraphs, front-loaded with the core purpose, and every sentence provides meaningful detail (explicit vs computed cost, unpriced behavior). No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the primary behavior and a key edge case (unpriced model), while return format is covered by the output schema. It does not mention required fields or validation, but those are already in the schema, so the description is sufficiently complete for the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description enriches the meaning of cost-related parameters (e.g., what happens if the model is unpriced) that the schema does not fully cover. However, it does not explain the overall `run` object structure or other parameters, and the top-level schema coverage is 0%, so compensation is partial.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action ('Record a single LLM/agent run') and the resource targeted, plus the outcome ('return the stored row'). This clearly distinguishes it from sibling tools like runmeter_list or runmeter_get, which read or summarize data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies that this tool is for logging new runs, but it does not explicitly state when to use it versus alternatives. No mention of prerequisites or exclusions, so the usage context is implied rather than directly guided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
runmeter_summaryARead-onlyIdempotent
Aggregate telemetry into cost and reliability rollups.
group_by is one of: 'model', 'agent', 'finish_reason', 'day', or 'tag'.
Optional since limits the window (ISO timestamp or relative like '30d').
Each group reports run count, total input/output tokens, total cost,
average latency, and error rate. Groups are sorted by total cost descending.
| Name | Required | Description | Default |
|---|---|---|---|
| since | No | ||
| group_by | No | model |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark the tool as read-only and idempotent. The description adds valuable behavioral details: group_by allowed values, since accepting ISO or relative formats, sorting by total cost descending, and the exact metrics reported per group. No contradictions are present.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded: purpose first, then parameter details, then output behavior. Every sentence adds relevant information without any fluff or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present and strong annotations, the description nevertheless covers purpose, parameters, and output metrics in sufficient detail. Minor semantics like timezone are not specified but are not critical for this simple aggregation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and parameters are plain strings, but the description fully compensates by listing all valid group_by values and explaining the since format (ISO timestamp or '30d'). This adds meaning well beyond the schema, though defaults are already present in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Aggregate telemetry into cost and reliability rollups,' which uses a specific verb and resource, and distinct output details. This clearly separates it from sibling tools like runmeter_get, runmeter_list, and runmeter_export, which are not aggregation tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for aggregate summary queries but never explicitly contrasts it with alternatives or states when not to use it. There is no mention of sibling tools or exclusions, so the guidance is implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
6 tool updates
v0.1.0- First observed
runmeter_delete - First observed
runmeter_export - First observed
runmeter_get - First observed
runmeter_list - First observed
runmeter_record - First observed
runmeter_summary
TDQS
Scored across 6 tools
Each tool targets a distinct operation: recording, retrieving a single run, listing with filters, aggregating summaries, exporting, and deleting. There is no overlap in purpose, making tool selection unambiguous.
All tool names follow the pattern 'runmeter_' + verb (get, summary, record, list, export, delete). The consistent prefix and verb-based naming make the toolset predictable and easy to navigate.
Six tools is well within the ideal 3-15 range and perfectly scoped for a telemetry server. Each tool earns its place, covering core operations without unnecessary bloat.
The toolset covers the full lifecycle of LLM/agent run telemetry: record, retrieve, list, analyze, export, and delete. There are no obvious missing operations for the domain, and the single-run delete with confirmation prevents accidental data loss.
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Connectors
MCP server for building and testing AI agents with multi-model experimentation and insights.
Hosted MCP server for LLM cost estimation, model comparison, and budget-aware routing.
MCP server providing access to the Scorecard API to evaluate and optimize LLM systems.
- SpanlyOAuthcom.spanly
MCP observability. Query live traffic, errors, duration, and alerts from your AI agent.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceA Model Context Protocol (MCP) server for comprehensive monitoring and observability of systems using Langfuse.1MIT
- AlicenseNot gradedqualityDmaintenanceA MCP server for tracking AI usage metrics and structured logs across applications. Monitor model calls, analyze usage patterns, track costs, and debug AI interactions.12MIT
- AlicenseNot gradedqualityAmaintenanceAn MCP server that gives AI agents observability over their own tool calls, enabling auditing, cost tracking, latency analysis, and alerting.MIT
- AlicenseAqualityBmaintenanceMCP server for AI agent observability, providing trace and span logging, search, latency/tokens/cost metrics, and anomaly detection using an in-memory buffer.638MIT
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/Aptica-Solutions/a-mcp-runmeter'
If you have feedback or need assistance with the MCP directory API, please join our Discord server