Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@Warpmetrics MCP Server What is my total LLM spend for the last 7 days?"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Connect AI assistants to your Warpmetrics AI agent telemetry via the Model Context Protocol.
npm install -g @warpmetrics/mcpRelated MCP server: Datadog MCP Server
Create an API key at warpmetrics.com/app/api-keys
Add to your ~/.claude/claude_desktop_config.json:
{
"mcpServers": {
"warpmetrics": {
"command": "warpmetrics-mcp",
"env": {
"WARPMETRICS_API_KEY": "wm_live_your_api_key_here"
}
}
}
}The Warpmetrics tools will now be available.
Tools are loaded dynamically from the Warpmetrics API and stay automatically in sync.
View all available tools:
Documentation: warpmetrics.com/docs/mcp
CLI: npx @warpmetrics/mcp --list-tools
"How many runs did I have today?"
"Show me the most expensive calls"
"List recent failed runs"
"Show details for run wm_run_01abc123"
"What's my total LLM spend this week?"
"What's the average latency for my code-review agent?"
"Show me cost trends for the last 7 days"
"What's the success rate for my code-review agent?"
"Show me outcome statistics"
"List all outcome classifications"
Variable | Required | Description |
| Yes | Your Warpmetrics API key |
| No | API URL (default: https://api.warpmetrics.com) |
git clone https://github.com/warpmetrics/mcp.git
cd mcp
npm install
# Run locally
WARPMETRICS_API_KEY=wm_live_... node src/index.jsMIT
Get act detail. Retrieve a single act by ID with full details including the referenced outcome, target entity, and follow-up run.
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Act ID |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, but the description discloses that the response includes outcome, target entity, and follow-up run, providing behavioral context beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences, front-loaded with the main purpose, no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter retrieval tool with no output schema, the description adequately specifies what is included in the response.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear parameter description. The description does not add significant new meaning beyond 'single act by ID'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves a single act by ID with full details, including referenced objects, distinguishing it from siblings like list_acts.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when full detail for a specific act is needed, but does not explicitly state when not to use or mention alternatives like get_act_stats.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get act statistics. Retrieve aggregated act statistics grouped by name, optionally filtered by date range.
| Name | Required | Description | Default |
|---|---|---|---|
| from | No | Start date (ISO 8601) | |
| to | No | End date (ISO 8601) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions 'aggregated act statistics grouped by name' and optional date filter, but does not disclose whether it's read-only, performance implications, or what aggregation method is used (count, sum, etc.).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no unnecessary words. Front-loads the purpose and adds filtering detail efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description should clarify return structure (e.g., fields like counts, averages) but only mentions 'aggregated act statistics grouped by name', leaving the output ambiguous for agents.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for 'from' and 'to' as ISO 8601 dates. The description adds little beyond confirming optional date filtering. Baseline of 3 applies as schema already documents parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and resource 'act statistics', specifying grouping by name and optional date filtering, which distinguishes it from siblings like 'get_act' (single act) and 'get_stats' (general stats).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving aggregated stats by name with date filtering, but provides no explicit guidance on when to use this tool over siblings (e.g., get_outcome_stats, get_stats) or any exclusions/prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get call detail. Retrieve a single LLM call by ID with full details including messages, response, tools, and ancestry.
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Call ID |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses the contents of the call (messages, response, tools, ancestry), which is helpful. It does not explicitly state read-only nature, but that is clear from 'get' context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that immediately conveys the tool's purpose. No wasted words; it is efficiently structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple get tool with one parameter and no output schema, the description adequately explains what is returned. Missing details like error handling or rate limits are not critical given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 100% description coverage for the single parameter 'id'. The description reinforces the parameter's role by stating 'by ID', adding value beyond the schema's simple 'Call ID' description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'get' and resource 'call', and specifies the full details included (messages, response, tools, ancestry), distinguishing it from siblings like list_calls.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving a single call by ID, which contrasts with list_calls for listing. No explicit when-not or alternative guidance is given, but the context of siblings supports correct selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get global counters. Retrieve all-time counters for the project: total runs, groups, calls, outcomes, tokens, cost, and duration.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, but description clearly indicates a read operation ('Retrieve') and lists the returned counters. Does not mention side effects, authentication, or rate limits, though the simplicity of the tool reduces concern.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences that front-load the tool's purpose and immediately list the data returned. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless tool with no output schema, the description adequately lists all retrieved counters. Agent can understand exactly what data to expect without additional details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has zero parameters, so schema coverage is 100%. Description adds value by naming the specific counters returned, compensating for the lack of output schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Verb 'Get' clearly indicates retrieval, and 'global counters' with explicit listing of what counters (runs, groups, etc.) makes the tool's purpose specific. It distinguishes from sibling tools that focus on single entities or filtered data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Describes retrieval of all-time global counters, implying it is for high-level project summaries. Does not explicitly state when not to use or mention alternatives, but context suggests it is the correct choice for aggregate data.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get group detail. Retrieve a single group by ID with full details including calls and outcomes.
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Group ID |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries full burden. It discloses that the response includes calls and outcomes, but does not cover permissions, side effects, or rate limits. Acceptable but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no redundancy. Front-loaded with the purpose, then specifics. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple getter with one parameter and no output schema, the description sufficiently covers what the tool does and what data it returns. Could mention pagination or limits, but not critical.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter 'id' is already fully described in the schema (100% coverage). The description adds no additional meaning, thus baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb (get/retrieve) and resource (group detail). It specifies retrieval by ID and mentions included data (calls and outcomes), distinguishing it from sibling tools like list_groups.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving a single group with full details, but does not explicitly state when to use this over alternatives like list_groups or other get_* tools. No exclusions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get outcome detail. Retrieve a single outcome by ID with full details including acts, provenance chain, target resolution, and classification.
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Outcome ID |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses that it retrieves a single outcome by ID and lists the return fields, but does not mention error handling, auth requirements, or idempotency. Adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences front-load the purpose and expand with details. No unnecessary words. Highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description provides useful details about what is returned (acts, provenance chain, etc.). Complexity is low, and the description is mostly complete, though it could mention error behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with parameter 'id' described as 'Outcome ID'. The description adds no additional meaning beyond that, resulting in a baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves a single outcome by ID with full details including specific fields like acts, provenance chain, target resolution, and classification. This distinguishes it from sibling tools like list_outcomes (listing) or get_outcome_stats (statistics).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The context implies usage for fetching a specific outcome's details, but no explicit guidance on when to use versus alternatives. The naming and sibling differentiation provide clear context, so it scores 4.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get outcome statistics. Retrieve aggregated outcome statistics grouped by name, optionally filtered by date range.
| Name | Required | Description | Default |
|---|---|---|---|
| from | No | Start date (ISO 8601) | |
| to | No | End date (ISO 8601) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description does not disclose side effects, permissions, or behavior beyond basic aggregation, leaving gaps for an agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences efficiently convey purpose and optional filter with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple retrieval tool with two optional params, but lacks details on output format, grouping behavior, and scope (e.g., default date range).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear parameter descriptions; the description adds no additional semantics beyond restating the date range filter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb (get/retrieve) and resource (outcome statistics grouped by name), distinguishing it from siblings like get_outcome or get_stats.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage for aggregated outcome stats with optional date filter, but no explicit guidance on when to use vs alternatives or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get run detail. Retrieve a single run by ID with full details including groups, calls, outcomes, and totals.
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Run ID |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states 'retrieve', implying a read-only operation, but does not disclose any other behavioral traits like permissions or side effects. Adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. The first sentence serves as a clear title, the second adds specifics. Front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple retrieval tool with one parameter and no output schema, the description covers the purpose and what is returned. It could benefit from mentioning that it returns the full run object, but it is largely complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 100% description coverage for the single parameter 'id'. The description adds context by emphasizing 'by ID' and 'single run', reinforcing the parameter's role beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves a single run by ID with full details. It distinguishes itself from sibling tools like list_runs (which returns multiple runs) and get_run_matrix (which returns a matrix view).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states 'by ID' and that it's for a single run, providing clear context for when to use this tool over others. However, it does not mention when not to use it or provide alternatives explicitly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get run matrix. Retrieve a comparison matrix for runs with a given label, showing step-level metrics across runs.
| Name | Required | Description | Default |
|---|---|---|---|
| label | Yes | Label (required) | |
| from | No | Start date (ISO 8601) | |
| to | No | End date (ISO 8601) | |
| limit | Yes | Items per page | |
| offset | Yes | Pagination offset |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully convey behavior. It indicates a read operation ('retrieve') and a matrix output, but lacks details on pagination, cost, or side effects. The behavior is only superficially described.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description has two sentences, but the first ('Get run matrix') is redundant with the tool name. The second sentence is useful. It is somewhat wasteful but not overly long.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description does not explain the output format or details of the matrix (rows vs columns, metrics). With no output schema and 5 parameters, more context is needed for an agent to fully understand what is returned and how to use the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are already documented. The description adds no new semantic meaning beyond the schema for each parameter, but it does tie the label parameter to the tool's purpose. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves a comparison matrix for runs with a given label, showing step-level metrics. This verb+resource combination is specific and distinguishes it from siblings like get_run (single run) or list_runs (list of runs).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implicitly suggests use when comparing multiple runs by label, but it does not explicitly exclude alternatives or provide conditions for when not to use this tool. There is no mention of when to prefer get_run_timeline or list_runs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get run timeline. Retrieve the execution timeline for a run, showing the sequence of calls and groups with timing data.
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Run ID |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and description only says 'Retrieve' without disclosing behavioral traits like read-only safety, authorization needs, or rate limits. Agent lacks critical behavioral context for a data retrieval tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, 20 words, clear and front-loaded. No wasted words, but could incorporate more behavioral context without sacrificing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool is simple (1 param, no output schema or nested objects). Description explains what it returns (timeline with calls/groups/timing), sufficient for basic usage. Lacks mention of ordering or format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameter descriptions (id as Run ID). Description adds no extra meaning beyond schema, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Get run timeline' and explains it retrieves execution timeline with sequence of calls and groups and timing data. This distinguishes it from siblings like get_run (run metadata) and get_run_matrix (performance matrix).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage context is implied (when needing execution timeline), but no explicit guidance on when to use vs alternatives, no exclusions or prerequisites mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get summary statistics. Retrieve summary statistics including totals, trends vs previous period, and previous period values. Optionally filtered by date range.
| Name | Required | Description | Default |
|---|---|---|---|
| from | No | Start date (ISO 8601) | |
| to | No | End date (ISO 8601) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description must carry behavioral info. It adds what stats are included but lacks details on read-only nature, auth needs, error behavior, or output format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded purpose. First sentence is slightly redundant but overall concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema or annotations; description should explain return structure and scope (global vs project). Missing info about output format and context among siblings.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage 100%, so baseline 3. Description only reinforces date range filtering without adding new meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clear verb+resource: 'get summary statistics' with specifics (totals, trends). But lacks specificity to differentiate from siblings like get_act_stats or get_global_counters.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or alternatives. Only mentions optional date filtering, but no guidance on when to use this overall stats tool vs specific ones.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get time series. Retrieve time series data with automatic hourly/daily resolution. Returns runs, calls, cost, duration, and success/failure per bucket.
| Name | Required | Description | Default |
|---|---|---|---|
| from | No | Start date (ISO 8601) | |
| to | No | End date (ISO 8601) | |
| label | No | Filter by run label |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the full burden. It discloses automatic resolution selection (hourly/daily) and return data shape, but omits behavioral details like whether data is read-only, whether it requires specific permissions, or pagination behavior. Adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences; the first repeats the name but the second adds substantive detail. No unnecessary words, though the first sentence could be merged. Efficient overall.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and no annotations, the description partially compensates by listing returned metrics. However, it doesn't explain how resolution is determined (automatic based on range? parameter?), nor does it mention limits or defaults. Adequate for a simple retrieval tool but leaves questions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so each parameter already has a description. The tool description adds no new meaning beyond what the schema provides (e.g., it doesn't clarify format expectations for ISO 8601 or how label filtering works). Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves time series data with automatic resolution and lists the returned fields (runs, calls, cost, duration, success/failure). This distinguishes it from sibling tools like get_stats or list_runs which do not mention time series bucketing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, no prerequisites, and no exclusion criteria. With many sibling tools (get_stats, get_global_counters), explicit use case differentiation is missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List acts. Retrieve a paginated list of acts with optional filtering by name and date range.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | Yes | Items per page | |
| offset | Yes | Pagination offset | |
| name | No | Filter by act name | |
| hasFollowUp | No | Filter acts by whether they triggered a follow-up run | |
| from | No | Start date (ISO 8601) | |
| to | No | End date (ISO 8601) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description bears the full burden. It notes pagination and filtering but does not state that the operation is read-only, what data is returned, or any side effects. Safety and idempotency are only implied by the verb 'list'.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two short sentences with no extraneous information. The first sentence is somewhat tautological ('List acts'), but the second adds necessary context. It is concise, though it could be slightly more informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the basic listing and filtering capabilities, but it omits details about the response format (e.g., what fields each act object contains), ordering, and whether pagination metadata is returned. For a list tool with no output schema, these details would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so each parameter is already documented. The description adds value by grouping 'name' and 'date range' as filters, but it does not elaborate on pagination parameters (limit, offset) or the hasFollowUp filter. The baseline of 3 applies, and the description adds minimal extra meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'List acts' with optional filtering, clearly indicating the action and resource. However, it does not explicitly differentiate from sibling tools like list_calls or list_groups, though the resource name is distinct.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, nor does it mention prerequisites or restrictions. It only describes what it does without context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List calls. Retrieve a paginated list of LLM calls with optional filtering by date, model, status, and sort order.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | Yes | Items per page (max 100) | |
| offset | Yes | Pagination offset | |
| from | No | Start date (ISO 8601) | |
| to | No | End date (ISO 8601) | |
| model | No | Filter by model name | |
| status | No | Filter by status (success, error) | |
| sort | No | Sort field (e.g., cost, duration, endedAt) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It mentions pagination and filtering but does not state that the operation is read-only, nor does it address rate limits, authentication, or other behavioral traits. More transparency is needed for a safe tool usage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two sentences, with the first being a clear, front-loaded statement of purpose. The second adds needed details. It is concise and efficient, though could be slightly more precise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 7 parameters, 2 required, and no output schema. The description covers purpose and filtering but does not explain what the return structure looks like (e.g., pagination metadata, call objects). This leaves the agent guessing about the response format, which is a significant gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage with clear descriptions for all 7 parameters. The description's mention of filtering by date, model, status, and sort order is redundant with the schema. No additional semantic value is added beyond summarizing the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists calls and retrieves paginated LLM call data. It specifies filtering options, making the purpose distinct from sibling list tools like list_acts or list_runs. However, it does not explicitly differentiate itself from siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving call lists with optional filters, but provides no guidance on when to use this tool versus alternatives, or when not to use it. There are no exclusions or context for appropriate scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List classifications. Retrieve all outcome classification rules for the project.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the burden. It only states it retrieves all rules for the project, lacking details on side effects, permissions, or performance. Minimal disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two concise sentences with no wasted words. It is front-loaded with the action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with no parameters and no output schema, the description is minimally adequate. However, it does not describe the format of the returned classifications, which could aid agent understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so the input schema coverage is 100%. The description does not need to add parameter details, and with zero params the baseline is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists classifications and retrieves outcome classification rules for the project. It uses specific verbs and resource naming, and is distinct from sibling list_* tools like list_outcomes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. The description does not mention exclusions or context for selection among related list tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List groups. Retrieve a paginated list of groups with optional filtering by label and date range.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | Yes | Items per page (max 100) | |
| offset | Yes | Pagination offset | |
| label | No | Filter by label | |
| from | No | Start date (ISO 8601) | |
| to | No | End date (ISO 8601) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description indicates the tool is read-only (list operation) and supports pagination and filtering, which are the main behavioral traits. However, with no annotations provided, the description carries full burden and could be more explicit about side effects, authorization needs, or rate limits, though none are expected for a simple list.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise with two sentences. The first sentence states the primary action, and the second adds key details. No superfluous content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with no output schema, the description adequately covers the purpose, pagination, and filtering. It could briefly mention the return format (e.g., 'returns an array of group objects'), but the absence is not critical given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% description coverage for all 5 parameters. The description mentions filtering by label and date range, which maps to parameters, but adds no additional context beyond what the schema already provides. Thus, score is at baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List'), resource ('groups'), and key features (paginated, optional filtering by label and date range). It distinguishes from the sibling 'get_group' tool, which retrieves a single group.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when a list of groups is needed, but does not explicitly state when to use this tool versus alternatives (e.g., 'get_group' for a single group). No exclusions or when-not-to-use guidance is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List outcomes. Retrieve a paginated list of outcomes with optional filtering by name, classification, and date range.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | Yes | Items per page (max 100) | |
| offset | Yes | Pagination offset | |
| name | No | Filter by outcome name | |
| classification | No | Filter by classification (success, failure, neutral) | |
| from | No | Start date (ISO 8601) | |
| to | No | End date (ISO 8601) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully bears the burden. It discloses pagination and filtering but does not explicitly state that the operation is read-only or idempotent, nor does it mention authentication or rate limits. However, it is largely transparent about the core behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short and front-loaded with 'List outcomes', but contains slight redundancy (e.g., both 'List outcomes' and 'Retrieve a paginated list'). Still efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers filtering and pagination but lacks details about the response structure (what each outcome contains) and pagination iteration, which are important given no output schema. Adequate but could be more complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds marginal value by grouping parameters (filtering by name, classification, and date range) but does not provide new information beyond schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the verb 'list' and the resource 'outcomes', and specifies pagination and filtering capabilities. It is clear and distinguishes from sibling tools like get_outcome (single outcome) and get_outcome_stats (statistics).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving multiple outcomes with filtering but does not specify when to use this tool versus alternatives like get_outcome. No explicit guidance on when not to use it or comparison with siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List projects. List all projects accessible with this API key. Requires an org-scoped API key (sk_live_org_*).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Discloses the API key requirement, but lacks information about response format, pagination, or rate limits. Adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences that front-load the core action and add essential constraint. No redundant words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description should hint at return format or structure. It only states it lists projects but does not describe what the response contains. Sufficient for a simple list but incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has no parameters (100% coverage), so baseline is 4. Description does not need to add parameter information, and it doesn't.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'List projects' with a verb and resource, and distinguishes from siblings by specifying the resource 'projects'. The additional context about API key scope further clarifies purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Specifies the prerequisite of an org-scoped API key, which is critical for correct usage. Does not explicitly mention when not to use or alternatives, but the sibling tools are for different resources, so context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List run labels. Retrieve all distinct run labels for the project with aggregated stats, optionally filtered by date range.
| Name | Required | Description | Default |
|---|---|---|---|
| from | No | Start date (ISO 8601) | |
| to | No | End date (ISO 8601) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description lacks behavioral details such as read-only nature, authorization requirements, or side effects. The mention of 'aggregated stats' is vague and does not specify what stats are included.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, concise, and front-loaded with key information. Every word serves a purpose without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description mentions 'aggregated stats' but does not explain what statistics are returned or the response structure, leaving some context incomplete for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear date descriptions. The description adds value by reiterating optional filtering and introducing 'aggregated stats', which is not in the schema, slightly enhancing parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and resource 'run labels', and adds that it retrieves distinct labels with aggregated stats and optional date range filtering, distinguishing it from sibling list tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage with optional date filtering but does not provide explicit guidance on when to use this tool versus siblings like list_runs or other list tools, nor does it mention when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List runs. Retrieve a paginated list of runs with optional filtering by label and date range.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | Yes | Items per page (max 100) | |
| offset | Yes | Pagination offset | |
| label | No | Filter by label | |
| name | No | Filter by run name (from opts) | |
| hasOutcome | No | Filter runs by whether they have outcomes | |
| rootOnly | No | Filter to root runs only (exclude follow-up runs) | |
| from | No | Start date (ISO 8601) | |
| to | No | End date (ISO 8601) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description only states basic functionality (list, paginate, filter) without disclosing permissions, rate limits, or result behavior. Minimal behavioral insight.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences front-load the purpose and key functionality; no extraneous text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema provided; description does not detail return format, fields, or ordering. For a list tool with 8 parameters, additional context on pagination behavior or result structure is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all 8 parameters with descriptions (100% coverage); description mentions label and date range but adds no extra semantic value beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists runs and mentions pagination and filtering options, but does not differentiate from sibling list tools like list_acts or list_calls.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies use when a paginated list of runs is needed with optional filters, but lacks explicit when-not-to-use or alternatives like get_run for single runs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.0get_actget_act_statsget_outcomelist_actslist_calls1 field changedInput schema / properties / sort / descriptionPrevious value: -"Sort field (e.g., cost, latency, timestamp)"New value: +"Sort field (e.g., cost, duration, endedAt)"list_projectslist_runs3 fields changedInput schema / properties / hasOutcomeAdded value: +{
+ "description": "Filter runs by whether they have outcomes",
+ "type": "boolean"
+}Input schema / properties / nameAdded value: +{
+ "description": "Filter by run name (from opts)",
+ "type": "string"
+}Input schema / properties / rootOnlyAdded value: +{
+ "description": "Filter to root runs only (exclude follow-up runs)",
+ "type": "boolean"
+}v0.0.8get_callget_global_countersget_groupget_outcome_statsget_runget_run_matrixget_run_timelineget_statsget_timeserieslist_callslist_classificationslist_groupslist_outcomeslist_run_labelslist_runsScored across 20 tools
Each tool targets a distinct entity or specific view (e.g., get_act vs get_act_stats vs list_acts), with clear descriptions that eliminate ambiguity. No two tools overlap in purpose.
All tools follow the verb_noun pattern with 'get_' for singular retrieval and 'list_' for paginated lists, using consistent snake_case for compound nouns like 'run_matrix' and 'outcome_stats'.
20 tools is well-scoped for a read-only observability API covering runs, groups, calls, acts, outcomes, stats, and more. Each tool earns its place without redundancy.
The tool set covers most read operations for all entities, including specialized views like run matrix and time series. Minor gap: missing a 'get_classification' endpoint, but list_classifications mitigates this.
Query your Betterlytics web analytics from AI agents: traffic, funnels, journeys, errors, uptime.
Agent-Native Amplitude/Mixpanel - connect data sources, prompt for charts
MCP observability. Query live traffic, errors, duration, and alerts from your AI agent.
Analytics your AI agent can actually use. Track, experiment, and optimize via MCP.
