Skip to main content
Glama
metrxbots

Metrx MCP Server

by metrxbots

Metrx MCP Server

npm version CI License: MIT Smithery Glama

Your AI agents are wasting money. Metrx finds out how much, and fixes it.

The official MCP server for Metrx — the AI Agent Cost Intelligence Platform. Give any MCP-compatible agent (Claude, GPT, Gemini, Cursor, Windsurf) the ability to track its own costs, detect waste, optimize model selection, and prove ROI.

Why Metrx?

Problem

What Metrx Does

No visibility into agent spend

Real-time cost dashboards per agent, model, and provider

Overpaying for LLM calls

Provider arbitrage finds cheaper models for the same task

Runaway costs

Budget enforcement with auto-pause when limits are hit

Wasted tokens

Cost leak scanner detects retry storms, context bloat, model mismatch

Can't prove AI ROI

Revenue attribution links agent actions to business outcomes

Related MCP server: Odoo MCP Server

Quick Start

Try it now — no signup required

npx @metrxbot/mcp-server --demo

This starts the server with sample data so you can explore all 23 tools instantly.

Connect your real data

Option A — Interactive login (recommended):

npx @metrxbot/mcp-server --auth

Opens your browser to get an API key, validates it, and saves it to ~/.metrxrc so you never need to set env vars.

Option B — Environment variable:

METRX_API_KEY=sk_live_your_key_here npx @metrxbot/mcp-server --test

Get your free API key at app.metrxbot.com/sign-up.

Add to your MCP client (Claude Desktop, Cursor, Windsurf)

If you used --auth, no env block is needed — the key is read from ~/.metrxrc automatically:

{
  "mcpServers": {
    "metrx": {
      "command": "npx",
      "args": ["@metrxbot/mcp-server"]
    }
  }
}

Or pass the key explicitly via environment:

{
  "mcpServers": {
    "metrx": {
      "command": "npx",
      "args": ["@metrxbot/mcp-server"],
      "env": {
        "METRX_API_KEY": "sk_live_your_key_here"
      }
    }
  }
}

Remote HTTP endpoint

For remote agents (no local install needed):

POST https://metrxbot.com/api/mcp
Authorization: Bearer sk_live_your_key_here
Content-Type: application/json

From npm

npm install @metrxbot/mcp-server

23 Tools Across 10 Domains

Dashboard (3 tools)

Tool

Description

metrx_get_cost_summary

Comprehensive cost summary — total spend, call counts, error rates, and optimization opportunities

metrx_list_agents

List all agents with status, category, cost metrics, and health indicators

metrx_get_agent_detail

Detailed agent info including model, framework, cost breakdown, and performance history

Optimization (4 tools)

Tool

Description

metrx_get_optimization_recommendations

AI-powered cost optimization recommendations per agent or fleet-wide

metrx_apply_optimization

One-click apply an optimization recommendation to an agent

metrx_route_model

Model routing recommendation for a specific task based on complexity

metrx_compare_models

Compare LLM model pricing and capabilities across providers

Budgets (3 tools)

Tool

Description

metrx_get_budget_status

Current status of all budget configurations with spend vs. limits

metrx_set_budget

Create or update a budget with hard, soft, or monitor enforcement

metrx_update_budget_mode

Change enforcement mode of an existing budget or pause/resume it

Alerts (3 tools)

Tool

Description

metrx_get_alerts

Active alerts and notifications for your agent fleet

metrx_acknowledge_alert

Mark one or more alerts as read/acknowledged

metrx_get_failure_predictions

Predictive failure analysis — identify agents likely to fail before it happens

Experiments (3 tools)

Tool

Description

metrx_create_model_experiment

Start an A/B test comparing two LLM models with traffic splitting

metrx_get_experiment_results

Statistical significance, cost delta, and recommended action

metrx_stop_experiment

Stop a running model routing experiment and lock in the winner

Cost Leak Detector (1 tool)

Tool

Description

metrx_run_cost_leak_scan

Comprehensive 7-check cost leak audit across your entire agent fleet

Attribution (3 tools)

Tool

Description

metrx_attribute_task

Link agent actions to business outcomes for ROI tracking

metrx_get_task_roi

Calculate return on investment for an agent — costs vs. attributed outcomes

metrx_get_attribution_report

Multi-source attribution report with confidence scores and top contributors

Alert Configuration (1 tool)

Tool

Description

metrx_configure_alert_threshold

Set cost or operational alert thresholds with email, webhook, or auto-pause

ROI Audit (1 tool)

Tool

Description

metrx_generate_roi_audit

Board-ready ROI audit report for your AI agent fleet

Upgrade Justification (1 tool)

Tool

Description

metrx_get_upgrade_justification

ROI report for tier upgrades based on current usage patterns

Prompts

Pre-built prompt templates for common workflows:

Prompt

Description

analyze-costs

Comprehensive cost overview — spend breakdown, top agents, optimization opportunities

find-savings

Discover optimization opportunities — model downgrades, caching, routing

cost-leak-scan

Scan for waste patterns — retry storms, oversized contexts, model mismatch

Examples

"How much am I spending?"

User: What was my AI cost this week?
→ metrx_get_cost_summary(period_days=7)

Total Spend: $234.56 | Calls: 2,450 | Error Rate: 0.2%
├── customer-support: $156.23 (1,800 calls)
└── code-generator: $78.33 (650 calls)

💡 Switch customer-support from GPT-4 to Claude Sonnet: Save $42/week

"Find me savings"

User: Am I overpaying for my agents?
→ metrx_compare_models(models=["gpt-4o", "claude-3-5-sonnet", "gemini-1.5-pro"])

Model Comparison (per 1M tokens):
├── gpt-4o: $2.50 in / $10.00 out
├── claude-3-5-sonnet: $3.00 in / $15.00 out
└── gemini-1.5-pro: $3.50 in / $10.50 out

"Test a cheaper model"

User: Test Claude 3.5 Sonnet against my GPT-4 setup
→ metrx_create_model_experiment(agent_id="agent_123",
    model_a="gpt-4o", model_b="claude-3-5-sonnet-20241022", traffic_split=10)

Experiment started: 90% GPT-4o, 10% Claude 3.5 Sonnet
Check back in 14 days for statistical significance.

Companion Tool: Cost Leak Detector

This repo also includes @metrxbot/cost-leak-detector — a free, offline CLI that scans your LLM API logs for wasted spend. No signup, no cloud, no data leaves your machine.

npx @metrxbot/cost-leak-detector demo

It runs 7 checks (idle agents, premium model overuse, missing caching, high error rates, context overflow, no budgets, arbitrage opportunities) and gives you a scored report in seconds. See the full docs.

Configuration

API Key (required)

The server looks for your API key in this order:

  1. METRX_API_KEY environment variable

  2. ~/.metrxrc file (created by --auth)

Run npx @metrxbot/mcp-server --auth to save your key, or set the env var directly.

Variable

Required

Description

METRX_API_KEY

Yes*

Your Metrx API key (get one free)

METRX_API_URL

No

Override API base URL (default: https://metrxbot.com/api/v1)

*Not required if you've run --auth — the key is read from ~/.metrxrc automatically.

CLI Flags

Flag

Description

--demo

Start with sample data — no API key or signup needed

--auth

Interactive login — opens browser, validates key, saves to ~/.metrxrc

--test

Verify your API key and connection

Rate Limiting

60 requests per minute per tool. For higher limits, contact support@metrxbot.com.

Development

git clone https://github.com/metrxbots/mcp-server.git
cd mcp-server
npm install
npm run typecheck
npm test

Contributing

See CONTRIBUTING.md for guidelines.

A Note on Naming

The product is Metrx (metrxbot.com). The npm scope is @metrxbot and the Smithery listing is metrxbot/mcp-server. The GitHub organization is metrxbots (with an s) because metrxbot was already taken on GitHub. If you see metrxbot vs metrxbots across platforms, they're the same project — just a GitHub namespace constraint.

License

MIT — see LICENSE.

💬 Feedback

Did Metrx work for you? We'd love to hear it — good or bad.

If you installed but hit a snag, tell us what happened — we read every report.

Available Tools

23 tools
metrx_acknowledge_alertAcknowledge AlertA
Idempotent

Mark one or more alerts as read/acknowledged. This removes them from the unread alerts list but preserves them in history. Do NOT use for resolving the underlying issue — take action on the alert first.

ParametersJSON Schema
NameRequiredDescriptionDefault
alert_idsYesAlert IDs to acknowledge

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds context beyond annotations: removes from unread list but preserves in history. Mentions idempotent nature implicitly. Could mention return value but not critical.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two efficient sentences, no redundancy. Front-loads action and adds important caveat.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter, it covers purpose and usage guidance well. Lacks return value indication but overall sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema already has 100% coverage with parameter description 'Alert IDs to acknowledge'. Description adds no further semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states it marks alerts as read/acknowledged, removes from unread list, and preserves history. Differentiates from similar actions like resolving issues.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly instructs not to use for resolving underlying issues and to take action first. Provides clear when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_apply_optimizationApply OptimizationA
Idempotent

Apply a one-click optimization recommendation to an agent. Only works for suggestions marked as "one_click: true". Common optimizations include setting max_tokens limits and switching models. Do NOT use for unvalidated changes — run create_model_experiment first if unsure about impact.

ParametersJSON Schema
NameRequiredDescriptionDefault
agent_idYesThe agent to apply the optimization to
optimization_typeYesThe type of optimization to apply (e.g., "token_guardrails", "model_switch")
payloadNoOverride the default optimization payload (advanced)

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations are consistent (idempotentHint=true, readOnlyHint=false). Description adds critical behavioral context: the tool only applies one-click recommendations and should not be used for unvalidated changes. No contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences: purpose, condition, and usage guidance. No filler, front-loaded with essential information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (3 params, no output schema), the description fully covers what it does, when it works, and when to use alternatives. No missing crucial details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description provides examples of optimizations (max_tokens, model switch) but does not add detail beyond schema descriptions for the parameters themselves.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Apply' and the resource 'optimization recommendation to an agent'. It specifies the condition 'one_click: true', distinguishing it from other tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly warns 'Do NOT use for unvalidated changes' and recommends an alternative tool 'create_model_experiment' for uncertain cases. Also states that it only works for one_click suggestions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_attribute_taskAttribute Task to OutcomeA

Link an agent task/event to a business outcome for ROI tracking. This creates a mapping between agent actions and measurable business results. Do NOT use for reading attribution data — use get_attribution_report or get_task_roi.

ParametersJSON Schema
NameRequiredDescriptionDefault
agent_idYesThe agent UUID to attribute
event_idNoOptional: specific event/task ID to attribute
outcome_typeYesType of outcome
outcome_sourceYesSource of the outcome data
value_centsNoOutcome value in cents
descriptionNoOptional description of the outcome

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnlyHint=false and destructiveHint=false, so the description's claim of creating a mapping is consistent but adds minimal behavioral context beyond what annotations provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no wasted words. The purpose is front-loaded, and the usage guideline is concisely appended.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with 6 parameters and no output schema, the description covers purpose and usage exclusions adequately. It could mention the return value or confirmation, but the schema and annotations fill many gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents each parameter well. The description does not add additional meaning beyond the schema's definitions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Link'), the resource ('agent task/event to a business outcome'), and the purpose ('for ROI tracking'). It distinguishes itself from sibling tools by explicitly mentioning alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when not to use this tool ('Do NOT use for reading attribution data') and provides specific alternative tools ('use get_attribution_report or get_task_roi').

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_compare_modelsCompare ModelsA
Read-onlyIdempotent

Compare LLM model pricing and capabilities across providers. Returns pricing per 1M tokens, context window sizes, batch/cache support, and cost savings estimates for switching from a current model to alternatives. Works without any usage data (Day 0 value). Do NOT use for agent-specific recommendations — use get_optimization_recommendations which factors in actual usage patterns.

ParametersJSON Schema
NameRequiredDescriptionDefault
current_modelNoCurrent model to compare against (e.g., "gpt-4o", "claude-sonnet-4-20250514")
tierNoCapability tier to filter alternatives
providerNoFilter to a specific provider (e.g., "openai", "anthropic", "google")

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds value by specifying that the tool returns cost savings estimates and works without usage data (Day 0). No contradictions; adds useful behavioral context beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: first states purpose and returns, second provides usage guidance with sibling alternative. No wasted words, front-loaded with key information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite no output schema, the description lists returned data (pricing, context window, savings) and the use case. Combined with annotations (readOnly), it provides a complete picture for usage. All relevant signals covered.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema coverage is 100%, so each parameter already has a description. The description adds overall context but does not significantly enhance parameter understanding beyond the schema. Baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool compares LLM model pricing and capabilities across providers, listing specific return values (pricing per 1M tokens, context window sizes, etc.) and highlighting 'Day 0 value'. This is a specific verb-resource pair that distinguishes it from siblings like get_optimization_recommendations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says 'Do NOT use for agent-specific recommendations' and directs to an alternative sibling tool (get_optimization_recommendations), providing clear when-to-use and when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_configure_alert_thresholdConfigure Alert ThresholdA
Idempotent

Set up cost or operational alert thresholds for a specific agent or org-wide. Alerts can trigger email notifications, webhooks, or automatically pause the agent. Use for real-time cost governance and operational safety. Thresholds run server-side automatically. Do NOT use for viewing current alerts — use get_alerts instead.

ParametersJSON Schema
NameRequiredDescriptionDefault
agent_idNoSpecific agent UUID to configure alerts for. Omit for org-wide alerts.
metricYesMetric to monitor
threshold_valueYesThreshold value. For costs: cents (e.g., 500000 = $5000). For rates: decimal (e.g., 0.1 = 10%). For latency: ms.
actionYesAction to trigger when threshold is breached

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds behavioral context beyond annotations: alerts can trigger email, webhook, or auto-pause, and thresholds run server-side. Annotations idempotentHint=true is consistent with configure action; no contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each adding distinct value: purpose, actions/behaviors, and exclusion. No wasted words; front-loaded with core purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers purpose, usage, behavior, and parameter implications. Minor gap: no mention of return value (since no output schema), but otherwise complete for a configuration tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description mentions actions and threshold values but largely repeats information already in parameter descriptions. It does not add significant new meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool sets up cost or operational alert thresholds for an agent or org-wide, using specific verbs and resource identification. It explicitly distinguishes from the sibling tool 'get_alerts'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit when-to-use ('Set up... thresholds') and when-not-to-use ('Do NOT use for viewing current alerts — use get_alerts instead'), plus the use case context ('real-time cost governance and operational safety').

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_create_model_experimentCreate Model ExperimentA

Start an A/B test comparing two LLM models for a specific agent. Routes a percentage of traffic to the treatment model and tracks cost, latency, error rate, and quality metrics. The experiment runs until statistical significance is reached or the max duration expires. Do NOT use for one-off model comparisons — use compare_models for static pricing data.

ParametersJSON Schema
NameRequiredDescriptionDefault
agent_idYesAgent to run the experiment on
nameYesHuman-readable experiment name
treatment_modelYesThe candidate model to test (e.g., "gpt-4o-mini", "claude-haiku-4-20250414")
traffic_pctNoPercentage of traffic to route to the treatment model (default: 10%)
primary_metricNoThe primary metric to optimize for (default: cost_per_call)cost_per_call
max_duration_daysNoMaximum experiment duration in days (default: 14)
auto_promoteNoAutomatically apply the winning model when the experiment completes

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses key behaviors: routing traffic, tracking metrics, running until significance or max duration, and auto-promote option. Annotations provide no behavioral hints, so the description carries the full burden. It is thorough but could mention what happens if not auto-promoted or how to stop. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences with no wasted words. It front-loads the main purpose, then explains mechanics, and ends with usage guidelines. Well-structured and concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (7 params, no output schema), the description covers purpose, behavior, and sibling distinction. It lacks mention of the return value (like experiment ID) and prerequisites (e.g., agent must exist). Still, it is fairly complete for an AI agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for all 7 parameters. The tool description summarizes the parameters' roles but does not add significant new meaning beyond the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Start an A/B test comparing two LLM models for a specific agent.' It also distinguishes from a sibling tool by explicitly saying 'Do NOT use for one-off model comparisons — use compare_models for static pricing data.'

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit when-to-use and when-not-to-use guidance, including an alternative tool for one-off comparisons. It says 'Routes a percentage of traffic...' and 'Do NOT use for one-off model comparisons — use compare_models for static pricing data.'

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_generate_roi_auditGenerate ROI Audit ReportA
Read-onlyIdempotent

Generate a comprehensive ROI audit report for your AI agent fleet. Includes per-agent cost/revenue breakdown, attribution confidence scores, optimization opportunities, and risk flags. Suitable for board reporting and compliance. Do NOT use for quick per-agent ROI checks — use get_task_roi for individual agents.

ParametersJSON Schema
NameRequiredDescriptionDefault
period_daysNoAnalysis period in days (7-365)
include_methodologyNoInclude methodology notes and caveats for auditors
agent_idsNoSpecific agent IDs to include. Omit for full fleet audit.

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds value by detailing the report contents (per-agent breakdown, confidence scores, risk flags) and suitability for board reporting, which provides context beyond the annotations about the nature of output and use cases.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise with two sentences: the first states the purpose and contents, the second provides explicit usage guidance with a sibling alternative. No extraneous text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the annotations cover safety and the schema covers parameters, the description provides sufficient context about the report's purpose and contents. However, it lacks details about the output format (e.g., JSON structure), but the listed contents partially compensate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description does not add additional meaning beyond the schema's parameter descriptions (e.g., period_days, include_methodology, agent_ids).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it generates a comprehensive ROI audit report for the AI agent fleet, listing specific contents (cost/revenue breakdown, confidence scores, opportunities, risk flags). It distinguishes itself from the sibling tool get_task_roi by explicitly stating when not to use and providing the alternative.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states the appropriate use cases (board reporting, compliance) and provides a clear exclusion: 'Do NOT use for quick per-agent ROI checks — use get_task_roi for individual agents.' This directly guides the agent on tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_get_agent_detailGet Agent DetailA
Read-onlyIdempotent

Get detailed information about a specific agent including its model, framework, category, outcome configuration, and failure risk score. Do NOT use for fleet-wide overviews — use get_cost_summary instead.

ParametersJSON Schema
NameRequiredDescriptionDefault
agent_idYesThe agent UUID to look up

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true, so the description reinforces that this is a safe read. It adds value by listing the kinds of information returned (model, framework, etc.), but does not mention error handling or response format, which is acceptable for a simple lookup.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: one defining the action and output fields, the second providing usage guidance. Efficient and front-loaded with essential information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple interface (one required param, no output schema), the description adequately describes the tool's purpose and output content. However, it does not specify the return structure (e.g., single object vs array), which would complete the picture.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the schema describes agent_id as 'The agent UUID to look up'. The description does not add additional parameter details beyond what the schema provides, so baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Get detailed information about a specific agent' and lists specific fields (model, framework, category, etc.), distinguishing it from the sibling tool get_cost_summary which is for fleet-wide overviews.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly tells when NOT to use ('Do NOT use for fleet-wide overviews') and directs to the correct alternative ('use get_cost_summary instead'). This provides clear context for selection among siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_get_alertsGet AlertsA
Read-onlyIdempotent

Get active alerts and notifications for your agent fleet. Includes cost spikes, error rate increases, budget warnings, and system health notifications. Optionally filter by severity. Do NOT use for configuring alert triggers — use configure_alert_threshold for that.

ParametersJSON Schema
NameRequiredDescriptionDefault
severityNoFilter by alert severity
unread_onlyNoOnly return unread alerts (default: true)
limitNoMaximum number of alerts to return

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already indicate the tool is read-only, non-destructive, and idempotent. The description adds no behavioral details beyond that, but it does not contradict the annotations. It explains what the tool returns, which is sufficient given the annotations' clarity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise with three short sentences: the first states the purpose, the second lists alert types, and the third provides an exclusion and alternative. No extraneous information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple read-only tool with three optional parameters and no output schema, the description covers the core functionality, returned content, and key constraints. It is complete for its complexity level.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for all three parameters (severity, unread_only, limit), so the tool description does not need to add parameter details. The description only reiterates filtering by severity, which is already in the schema, thus no additional value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool retrieves active alerts and notifications for the agent fleet, listing specific alert types (cost spikes, error rate increases, budget warnings, system health). It explicitly distinguishes from the sibling tool for configuring alert triggers, ensuring no confusion.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides an explicit when-not-to-use scenario and directs users to an alternative tool (configure_alert_threshold) for configuring triggers. However, it does not elaborate on when to use this tool over other sibling tools for similar tasks, which would make it a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_get_attribution_reportGet Attribution ReportA
Read-onlyIdempotent

Get attribution report showing which agent actions led to business outcomes. Shows outcome counts, total values, confidence scores, and top contributing agents. Do NOT use for board-level reporting — use generate_roi_audit for formal audit reports.

ParametersJSON Schema
NameRequiredDescriptionDefault
agent_idNoOptional: filter to specific agent (omit for fleet-wide)
daysNoNumber of days to include (default: 30)
modelNoAttribution model preference. Note: the current /api/outcomes endpoint returns outcomes regardless of attribution method, so this is informational only. Canonical enum source: agentledger apps/web/lib/attribution/types.ts.direct

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only, non-destructive, idempotent behavior. The description adds transparency about the 'model' parameter being informational only due to backend behavior, and references the canonical enum source. This disclosure is valuable beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise: three sentences that front-load the purpose, include the warning about alternative tool use, and add necessary behavioral notes. No superfluous words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite lacking an output schema, the description hints at return structure (outcome counts, values, etc.). With zero required parameters and all schema descriptions covered, the contextual information is sufficient for most usage scenarios. Could mention pagination or limits, but not essential.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage, the schema already describes all three parameters. The description adds value by explaining the 'model' parameter's actual effect (informational only) and providing context about the enum source. The baseline is 3, and this added insight merits a 4.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool retrieves an attribution report linking agent actions to business outcomes, listing specific contents (outcome counts, values, confidence scores, top agents). It also distinguishes from the sibling 'generate_roi_audit' by explicitly excluding its use for board-level reporting.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance on when not to use this tool ('Do NOT use for board-level reporting') and directs to an alternative ('use generate_roi_audit'). It lacks an explicit 'when to use' statement but the context of attribution analysis is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_get_budget_statusGet Budget StatusA
Read-onlyIdempotent

Get the current status of all budget configurations. Shows spending vs limits, warning/exceeded counts, and enforcement modes. Use this to monitor spending governance across your agent fleet. Do NOT use for creating/changing budgets — use set_budget or update_budget_mode.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true. The description adds context about what the tool shows (spending vs limits, warning/exceeded counts, enforcement modes) and confirms it accesses all budget configurations, aligning with annotations with no contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: first states purpose and content, second provides usage guidance. No extraneous information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no parameters and a straightforward read operation, the description covers what the tool returns. It is complete enough for an agent to understand the tool's function, though it does not detail the output structure (no output schema provided).

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has zero parameters, so there is no need for parameter explanations. The description adds no parameter detail, but baseline 4 is appropriate per guidelines.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'Get' and resource 'budget status', clearly stating it retrieves current status of all budget configurations. It distinguishes itself from sibling tools like set_budget and update_budget_mode by explicitly noting it is read-only.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description tells the agent exactly when to use this tool ('to monitor spending governance') and explicitly warns against using it for creating/changing budgets, directing to specific alternatives ('set_budget or update_budget_mode').

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_get_cost_summaryGet Cost SummaryA
Read-onlyIdempotent

Get a comprehensive cost summary for your AI agent fleet. Returns total spend, call counts, error rates, agent breakdown, revenue attribution (if available), and optimization opportunities. Use this as the starting point for understanding your agent economics. Do NOT use for real-time per-request cost checking — use OpenTelemetry spans for that.

ParametersJSON Schema
NameRequiredDescriptionDefault
period_daysNoNumber of days to include in the summary (default: 30)

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnlyHint=true, destructiveHint=false, idempotentHint=true. Description adds value by listing returned fields (total spend, call counts, etc.), but doesn't mention response format or pagination. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no fluff. Purpose and usage guidance are front-loaded. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Return fields are listed, usage context provided, and tool differentiated from sibling. However, no output schema exists, and the description doesn't specify whether data is cached or real-time, leaving minor gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema has 100% coverage for the single parameter 'period_days' with its description. The tool description does not add any additional meaning or nuance to the parameter, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'Get a comprehensive cost summary for your AI agent fleet' – specific verb and resource. Distinguishes from sibling tools by referencing real-time per-request cost checking as a separate use case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly tells when to use ('starting point for understanding agent economics') and when not to use ('Do NOT use for real-time per-request cost checking'), and provides alternative ('use OpenTelemetry spans').

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_get_experiment_resultsGet Experiment ResultsA
Read-onlyIdempotent

Get the current results of a model routing experiment. Shows sample counts, metric comparisons, statistical significance, and the current winner (if determined). Do NOT use for starting experiments — use create_model_experiment.

ParametersJSON Schema
NameRequiredDescriptionDefault
agent_idNoFilter experiments by agent
statusNoFilter by experiment status

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true, so agent knows it's safe. Description adds what results contain (samples, metrics, significance, winner), which is useful but not critical for safety.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first states purpose and outputs, second provides negative guidance. No wasted words, immediately informative.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only result tool with good annotations and full schema coverage, the description covers the key outputs and usage boundary. Lacks mention of real-time vs cached data, but overall sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so description carries little burden. It does not elaborate on parameter meanings, but the schema descriptions are sufficient. No additional value beyond schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Get' and resource 'experiment results' and lists specific outputs (sample counts, metric comparisons, etc.). It also distinguishes from starting experiments, providing clear purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when not to use ('Do NOT use for starting experiments') and provides the alternative tool ('use create_model_experiment'). This is excellent guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_get_failure_predictionsGet Failure PredictionsA
Read-onlyIdempotent

Get predictive failure analysis for your agents. Shows upcoming risk of error rate breaches, latency degradation, cost overruns, rate limit risks, and budget exhaustion. Each prediction includes confidence level and recommended actions. Do NOT use for current/past failures — use get_alerts for active issues.

ParametersJSON Schema
NameRequiredDescriptionDefault
agent_idNoFilter predictions for a specific agent
severityNoFilter by prediction severity
statusNoFilter by prediction status (default: active)active

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, covering safety. Description adds context about output content (predictions with confidence and recommendations), which is not in schema. No contradictions, but could mention pagination or rate limits; still strong.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no wasted words. Front-loaded with purpose, followed by usage guidance. Structure is clear and efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only query tool with three optional parameters and no output schema, the description adequately explains what the tool returns (predictions with confidence and recommendations) and distinguishes from related tools. No missing critical information.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so schema already fully describes the three optional parameters (agent_id, severity, status). Description does not add parameter details beyond the schema, meeting the baseline for high coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Get' and resource 'predictive failure analysis for your agents', listing specific risk types (error rate breaches, latency degradation, etc.). It explicitly distinguishes from siblings by advising not to use for current/past failures and referencing get_alerts.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit guidance on when to use (upcoming risks) and when not to (current/past failures), with an alternative tool mentioned (get_alerts). Also notes that predictions include confidence level and recommended actions, aiding decision-making.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_get_optimization_recommendationsGet Optimization RecommendationsA
Read-onlyIdempotent

Get AI-powered cost optimization recommendations for a specific agent or your entire fleet. Returns actionable suggestions including model switching, token guardrails, provider arbitrage, batch processing opportunities, and revenue intelligence insights. Each suggestion includes estimated monthly savings and confidence level. Do NOT use for implementing fixes — use apply_optimization for one-click fixes or create_model_experiment to validate first.

ParametersJSON Schema
NameRequiredDescriptionDefault
agent_idNoSpecific agent to analyze. Omit for fleet-wide recommendations.
include_revenueNoInclude revenue-side insights (R3, R4, R6) in addition to cost optimizations

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint and idempotentHint. Description adds context about output content (estimated monthly savings, confidence level) and reinforces read-only nature, but no contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, no fluff. Front-loaded purpose, lists content, then provides usage boundaries. Efficient and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Describes output partially (suggestions with savings and confidence) and includes usage boundaries. Without output schema, this is adequate; minor gap in not stating exact structure.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so description adds no significant parameter info beyond existing schema descriptions. Baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'Get AI-powered cost optimization recommendations' with specific examples (model switching, provider arbitrage, etc.) and distinguishes from sibling tools like apply_optimization.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says 'Do NOT use for implementing fixes' and directs to apply_optimization or create_model_experiment, providing clear when-to-use and when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_get_task_roiGet Agent ROIA
Read-onlyIdempotent

Calculate return on investment for an agent. Shows total costs (LLM API calls), total outcomes (attributed business value), ROI multiplier, and breakdown by model and outcome type. Useful for identifying which agents generate the most value per dollar spent. Do NOT use for fleet-wide ROI — use generate_roi_audit for that.

ParametersJSON Schema
NameRequiredDescriptionDefault
agent_idYesThe agent UUID to calculate ROI for
daysNoNumber of days to analyze (default: 30)

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds value by detailing the output structure (costs, outcomes, breakdown), but does not reveal additional behavioral traits beyond what annotations imply. No contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first declares purpose, second lists output and exclusion. Front-loaded, no extraneous words, every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity, rich annotations, and no output schema, the description covers purpose, usage guidelines, and high-level output. Slightly incomplete on return format, but sufficient for an experienced user.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema has 100% description coverage, so baseline is 3. The description does not add extra meaning to parameters beyond the schema; it focuses on output rather than parameter details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool calculates ROI for an agent, listing specific metrics (costs, outcomes, multiplier, breakdown) and explicitly distinguishes from the sibling tool generate_roi_audit for fleet-wide analysis.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear when-to-use ('calculating ROI for an agent') and when-not-to-use ('Do NOT use for fleet-wide ROI'), with a direct sibling reference. Also notes usefulness for identifying high-value agents.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_get_upgrade_justificationGet Upgrade JustificationA
Read-onlyIdempotent

Generate an ROI report explaining why an upgrade from Starter to Lite/Pro tier makes sense. Analyzes current usage patterns, calculates optimization potential at higher tiers, and provides a structured upgrade recommendation with projected monthly savings. Do NOT use if already on Lite or Pro tier — not relevant for paid-tier users.

ParametersJSON Schema
NameRequiredDescriptionDefault
period_daysNoNumber of days to analyze for upgrade justification (default: 30)

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the tool as read-only, non-destructive, and idempotent. The description adds behavioral context: it analyzes usage patterns, calculates optimization potential, and provides a structured recommendation with projected savings. This goes beyond the annotations and informs the agent about the tool's output and processing.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences: the first covers purpose and functionality, the second gives a clear negative use case. No unnecessary words, well-structured with key information upfront.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one optional parameter and no output schema, the description covers purpose, process, and a usage guideline. It distinguishes from related siblings. A minor gap is the lack of detail on the output format, but it is still sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the only parameter (period_days). The description does not add any new semantics beyond what the schema already provides. According to the rubric, baseline is 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool generates an ROI report for upgrading from Starter to Lite/Pro tier. It uses a specific verb ('Generate') and resource ('ROI report'), and the context of 'upgrade justification' distinguishes it from sibling tools like metrx_generate_roi_audit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says 'Do NOT use if already on Lite or Pro tier — not relevant for paid-tier users.' This provides clear exclusion criteria. However, it does not name specific alternative tools for paid-tier users, which would elevate it to a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_list_agentsList AgentsA
Read-onlyIdempotent

List all AI agents in your organization with their status, category, and cost. Optionally filter by status or category. Returns agent IDs needed for other tools. Do NOT use for detailed per-agent analysis — use get_agent_detail for that.

ParametersJSON Schema
NameRequiredDescriptionDefault
statusNoFilter by agent status
categoryNoFilter by agent category (e.g., "sales", "support", "engineering")

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, so the description does not need to repeat safety. However, it adds valuable context that the tool 'Returns agent IDs needed for other tools', which informs the agent about how the output may be used downstream.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no unnecessary words. The key information (what the tool does and when to avoid it) is front-loaded. Every sentence serves a purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list tool with two optional parameters and no output schema, the description covers everything needed: what is returned, that IDs can be used elsewhere, and the boundary with the detail tool. Annotations handle safety and idempotency.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% (both parameters have descriptions). The description merely reiterates that filtering is optional ('Optionally filter by status or category') without adding new meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('List') and resource ('AI agents'), and clarifies what data is returned (status, category, cost). It also distinguishes from the sibling tool 'get_agent_detail' by stating that this is for listing, not detailed analysis.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when not to use this tool ('Do NOT use for detailed per-agent analysis') and provides a clear alternative ('use get_agent_detail for that'). This helps the agent select the correct tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_route_modelRoute Model SelectionA
Read-onlyIdempotent

Get a model routing recommendation for a specific task based on complexity. Uses the agent's historical performance data and cost analysis to suggest the optimal model for each task complexity level. Helps reduce costs by routing simple tasks to cheaper models while keeping complex tasks on premium models. Do NOT use for comparing all models at once — use compare_models for static pricing.

ParametersJSON Schema
NameRequiredDescriptionDefault
agent_idYesThe agent to get routing recommendations for
task_complexityYesEstimated task complexity: low (simple lookups/formatting), medium (analysis/summarization), high (reasoning/generation)
current_modelNoCurrently configured model (e.g., "gpt-4o"). If omitted, uses agent primary model.

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds behavioral context: it uses historical performance data and cost analysis, and suggests optimal model per complexity level. This complements the annotations well.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences effectively convey purpose, mechanism, and usage boundaries. No filler; each sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

While the description covers purpose and usage well, it lacks details about the tool's output (return value format). With no output schema, specifying what the recommendation looks like would be helpful.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description does not add new parameter-level details beyond what the schema provides, though it aligns with the task_complexity parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool gets a model routing recommendation for a specific task based on complexity, using historical data and cost analysis. It distinguishes itself from the sibling tool compare_models by explicitly excluding that use case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance on when to use this tool ('for a specific task based on complexity') and when not to ('Do NOT use for comparing all models at once'), directly naming the alternative tool (compare_models).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_run_cost_leak_scanRun Cost Leak ScanA
Read-onlyIdempotent

Run a comprehensive cost leak audit across your entire agent fleet. Identifies 7 types of cost inefficiencies: idle agents, model overprovisioning, missing caching, high error rates, context bloat, missing budgets, and cross-provider arbitrage opportunities (covers anthropic, cohere, google, mistral, openai, xai). Returns a scored report with fix recommendations and estimated monthly savings. Supports output_format="json" for machine-readable output in CI/CD pipelines. Do NOT use as a continuous monitoring loop — use configure_alert_threshold for ongoing monitoring. Do NOT use for fixing leaks — use apply_optimization for one-click fixes.

ParametersJSON Schema
NameRequiredDescriptionDefault
agent_idNoScan a specific agent instead of the entire fleet
include_low_severityNoInclude low-severity findings in the report
output_formatNoOutput format: "text" (default) returns a human-readable markdown report; "json" returns raw machine-readable JSON suitable for CI/CD pipelines and programmatic processing.text

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, destructiveHint, and idempotentHint. The description adds context about the 7 inefficiency types and output format options, but the behavioral safety is already well covered by annotations. The description does not contradict annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured paragraph. It front-loads the main purpose, then lists inefficiencies, output description, and usage caveats. Every sentence adds value with no repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description sufficiently describes the return value (scored report with fix recommendations and estimated savings). It also explains the output format options and the scope of the audit. The description is complete for an agent to understand tool behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and each parameter has a clear description. The description adds value by explaining the purpose of output_format for CI/CD pipelines and clarifying that agent_id is for scanning a specific agent. This goes beyond the schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it runs a comprehensive cost leak audit across the agent fleet, identifies 7 specific inefficiencies, and returns a scored report with recommendations. It distinguishes itself from sibling tools like configure_alert_threshold and apply_optimization by explicitly stating what it does and does not do.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit guidance: 'Do NOT use as a continuous monitoring loop — use configure_alert_threshold' and 'Do NOT use for fixing leaks — use apply_optimization'. This clearly tells the agent when to use this tool and when to use alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_set_budgetSet BudgetA
Idempotent

Create or update a budget configuration for an agent or the entire organization. Budgets enforce spending limits with configurable enforcement modes: "alert_only" (notify but don't block), "soft_block" (block with override), or "hard_block" (strict enforcement). Specify limits in dollars. Do NOT use just to change enforcement mode — use update_budget_mode for that.

ParametersJSON Schema
NameRequiredDescriptionDefault
agent_idNoAgent to set budget for. Omit for org-wide budget.
periodYesBudget period
limit_dollarsYesSpending limit in dollars (e.g., 100 for $100/month)
warning_pctNoPercentage of limit that triggers a warning (default: 80)
enforcement_modeNoHow to enforce the budget when exceededalert_only

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description explains the enforcement modes and the create/update behavior, adding context beyond annotations (readOnlyHint=false, idempotentHint=true). It does not contradict annotations. Minor gap: it does not explicitly state whether calling it overwrites all existing budget fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no wasted words. The core purpose is front-loaded, and every sentence serves a clear purpose (purpose + usage guidance).

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description does not mention return values or side effects. It adequately covers input parameters and usage restrictions, but the lack of response information is a gap for a tool that modifies state.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%. The description adds value by explaining enforcement modes with examples, clarifying the agent_id omission for org-wide budgets, and giving a dollar format example, all beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool creates or updates a budget configuration for an agent or organization, using specific verbs and resource. It also distinguishes itself from the sibling 'update_budget_mode' by explicitly warning against using it solely for mode changes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance on when not to use the tool (just to change enforcement mode) and points to the correct alternative (update_budget_mode). However, it does not discuss when to use this tool over other budget-related siblings like get_budget_status.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_stop_experimentStop ExperimentA
Idempotent

Stop a running model routing experiment. The experiment results are preserved. If the treatment model won, you can optionally promote it as the new default. Do NOT use for pausing experiments temporarily — stopping is permanent.

ParametersJSON Schema
NameRequiredDescriptionDefault
experiment_idYesThe experiment ID to stop
promote_winnerNoIf the treatment model won, apply it as the new default model

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations (idempotentHint=true, destructiveHint=false) align with description. Description adds that results are preserved and optional promotion, providing full behavioral disclosure beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, front-loaded with the action, and every sentence adds value without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (2 params, no output schema), the description fully covers purpose, usage, behavior, and parameter semantics. No gaps remain.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline 3. Description adds context to promote_winner ('If the treatment model won, you can optionally promote it'), clarifying its purpose beyond the schema default description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Stop a running model routing experiment' with a specific verb and resource. It distinguishes from pausing and mentions results preservation, differentiating from sibling tools like pausing actions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says 'Do NOT use for pausing experiments temporarily — stopping is permanent,' providing clear when-not-to-use guidance and an alternative context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metrx_update_budget_modeUpdate Budget ModeA
Idempotent

Change the enforcement mode of an existing budget or pause/resume it. Use "alert_only" for monitoring, "soft_block" for overridable limits, or "hard_block" for strict enforcement. Do NOT use to create new budgets — use set_budget for that.

ParametersJSON Schema
NameRequiredDescriptionDefault
budget_idYesThe budget configuration ID to update
enforcement_modeNoNew enforcement mode
pausedNoSet to true to pause the budget, false to resume

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate non-read-only, non-destructive, idempotent behavior. The description adds context about pausing/resuming and mode meanings but does not disclose potential side effects or error handling. It is consistent with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with purpose, then mode guidance, then exclusion. No superfluous words; each sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers the core functionality and usage boundaries. For a tool with three parameters and no output schema, it is largely complete. Could mention what happens on invalid budget_id, but this is acceptable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds value by explaining the three enum modes ('alert_only' for monitoring, etc.), which goes beyond the schema's minimal descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb (change/pause/resume) and the resource (budget), and distinguishes from sibling tool 'set_budget' for creation. It specifies the exact actions and modes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly tells when to use each mode ('alert_only for monitoring', etc.) and includes a clear exclusion: 'Do NOT use to create new budgets — use set_budget for that.' This provides strong guidance for the agent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

TDQS

A4.3/5.0
Disambiguation4/5

Most tools have distinct purposes with clear boundaries, such as get_alerts vs. configure_alert_threshold or compare_models vs. create_model_experiment. However, some overlap exists between get_optimization_recommendations and run_cost_leak_scan, as both identify cost inefficiencies, which could cause minor confusion for an agent.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern with the 'metrx_' prefix, such as metrx_acknowledge_alert, metrx_apply_optimization, and metrx_get_agent_detail. This uniformity makes the tool set predictable and easy to navigate.

Tool Count3/5

With 23 tools, the count is borderline high for a single server, potentially overwhelming. While the domain of AI agent cost and performance management is broad, the tool set feels dense, and some tools might be consolidated or split into more focused servers for better usability.

Completeness5/5

The tool set comprehensively covers the domain of AI agent management, including cost monitoring (e.g., get_cost_summary, set_budget), optimization (e.g., apply_optimization, run_cost_leak_scan), experimentation (e.g., create_model_experiment, stop_experiment), and reporting (e.g., generate_roi_audit, get_attribution_report). No obvious gaps are present, supporting full lifecycle management.

Maintenance

ActivityInactive
ResponsivenessSyncing

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    D
    maintenance
    An enterprise-grade MCP server for interacting with Odoo ERP through AI assistants, enabling guided workflows, native BI, and deep introspection.
    10
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    An MCP server that enables AI assistants to query databases, execute SQL, and manage Metabase resources like dashboards, cards, and collections through natural language.
    22
    MIT
  • A
    license
    Not graded
    quality
    B
    maintenance
    MCP server for Stessa rental-property accounting platform, enabling AI agents to query properties, portfolios, banking, documents, and tenancies via natural language.
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/metrxbots/mcp-server'

If you have feedback or need assistance with the MCP directory API, please join our Discord server