@argosvix/mcp-server

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v0.30.0-alpha.14
Disambiguation5/5
Every tool has a clearly distinct purpose. For example, there are multiple alert-related tools (acknowledge_alert, auto_silence_noisy_alert, silence_alert, etc.) but each targets a unique action like acknowledging a single event, bulk-silencing, or muting. Descriptions are detailed enough to avoid ambiguity.
Naming Consistency5/5
All 87 tool names follow a consistent verb_noun pattern in snake_case (e.g., create_alert, list_alerts, delete_alert). Verbs like get, list, create, update, delete, propose, etc. are used uniformly, making naming predictable.
Tool Count3/5
With 87 tools, the count is very high and exceeds typical well-scoped ranges (3-15). However, the server covers a broad domain (calls, alerts, budgets, evaluations, prompts, projects, etc.) as a comprehensive admin interface, so the count is somewhat justified but still feels heavy.
Completeness5/5
The tool set covers core CRUD/lifecycle operations for each subdomain (alerts, calls, eval, prompts, webhooks, budgets, policies, projects). There are no obvious gaps; even advanced operations like automated silencing, proposal reply, and approval request are included.
Average 4.4/5 across 87 of 87 tools scored. Lowest: 3.6/5.
See the Tool Scores section below for per-tool breakdowns.
- No community issues in the last 6 months
- 10 commits in the last 12 weeks
- No stable releases found
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI status not available
This repository is licensed under MIT License.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
This repository includes a glama.json configuration file.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavior. It correctly labels the operation as 'Retrieve' (read-only) and mentions filtering capabilities and default parameters. However, it omits details like pagination behavior (though present in schema) and potential limits, but overall it provides sufficient transparency for a read tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence but includes an undefined reference to 'tag' (not in input schema), which wastes space. It could be more structured (e.g., separating default behavior from filters). The front-loaded information is adequate.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 8 parameters and no output schema, the description is moderately complete. It covers the core purpose and defaults but does not mention pagination, ordering, or error conditions. The schema fills in many gaps, but the description could be more self-contained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds no semantic value beyond what the schema already provides for each parameter. It mentions 'tag' (not in schema), which is slightly misleading, but the defaults are reiterated.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves recent LLM call records. It specifies the resource ('call records captured by Argosvix'), the action ('Retrieve'), and key filtering dimensions (provider/model/time range/tag). It also distinguishes from sibling tools like aggregate_calls or export_calls by focusing on retrieval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving recent call records with filters, but it lacks explicit guidance on when to use this tool versus siblings (e.g., export_calls for bulk export, aggregate_calls for statistics). No when-not-to-use or alternative recommendations are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses the tool is non-executing ('nothing is executed'), which is key. However, it omits other details like authentication needs, rate limits, or side effects, leaving some transparency gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: two sentences and a brief instruction. No redundant words, front-loaded with the main action and key differentiator ('explanation only'). Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description is vague about the return value ('get the AI's reply'). Mentioning 'same as the inbox conversation' helps but doesn't specify format or structure. For a simple query tool, it is adequate but could be more complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both parameters. The description adds value by providing an example for 'body' (e.g., 'why did it degrade?') and a source hint for 'proposalId' ('from list_proposals'), improving agent understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action ('post a question about a proposal') and the resource ('proposal'), with a specific outcome ('get the AI's reply'). The 'same as the inbox conversation' and 'explanation only' differentiate it from sibling tools like get_proposal_thread and propose_eval_criteria, though not fully explicit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides a prerequisite ('Get proposalId from list_proposals') and a behavioral note ('nothing is executed'), but lacks explicit guidance on when to use or avoid this tool versus alternatives (e.g., get_proposal_thread for just viewing).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It states the tool reverses a silenced state, but does not disclose permissions, side effects, or behavior if the alert is not silenced. Adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single efficient sentence with no redundant information. It is front-loaded and concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter tool with no output schema, the description covers the core action adequately. However, it lacks context on post-unmuting behavior (e.g., notifications, state change effects).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers the single parameter 'alertId' with a description. The description does not add additional meaning beyond the schema, so baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (unmute) and resource (silenced alert). It distinguishes from siblings like 'silence_alert' and 'acknowledge_alert' by using specific verb and context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit guidance on when to use this tool versus alternatives, such as 'silence_alert' or 'acknowledge_alert'. It does not provide prerequisites (e.g., alert must be silenced) or when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses the limit of 50 and valid status values, but no annotations exist. Does not mention ordering, pagination, or whether it is read-only. The description partially informs behavior but leaves gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short, direct sentences. Front-loaded with action and key constraint. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple list tool but lacks description of output fields and pagination beyond the limit. No output schema exists, so the description should cover return value details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear enum and default. Description only restates the schema values, adding no new meaning beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action (List) and resource (approval requests) with a constraint (latest 50) and filter options. Distinguishes from siblings like create_approval or get_approval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies use for listing approvals with status filter, but lacks explicit guidance on when to use this tool versus other list tools (e.g., list_proposals) or conditions for using non-default status filters.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description fully bears the transparency burden. It only states deletion without mentioning reversibility, permissions needed, side effects, or response details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence plus the endpoint and scope, conveying essential information with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and no output schema, the description covers the basic purpose and scope, but lacks behavioral details like deletion permanence or success confirmation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (id parameter described with UUID). The description adds no additional meaning beyond what the schema already provides, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Delete' and the resource 'saved view with the given id', and distinguishes from sibling tools by being deletion-specific and scoped to account.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It mentions 'Scoped to your account', providing context for when to use, but does not explicitly state when not to use or name alternatives among the many sibling delete tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the behavioral burden. It mentions the 24-hour window for trigger history, which is a key behavioral trait, but lacks details on permissions, safety, or response format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, front-loaded with the core action, no filler. Every word serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with one optional parameter and no output schema, the description covers the essential context: what is returned and the time constraint. Minor omission of pagination or ordering but not critical.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the parameter description already covers the trigger history inclusion. The tool description mostly echoes the schema, adding no new semantic context beyond the time range.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Return' and clearly identifies the resource as 'configured alerts' and 'trigger history.' It distinguishes from siblings like get_alert (singular) and create_alert by explicitly stating it returns a list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not-to-use guidance is provided. It implies usage for listing alerts and recent history, but does not compare with alternatives like list_alert_events or acknowledge_alert.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. It reveals the endpoint and row structure but fails to state it is a read-only operation, lacks safety guarantees, and does not mention rate limits or authorization needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two precise sentences covering purpose, endpoint, row structure, and filtering. No extraneous information, making it highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although lacking output schema, the description details the fields in each row. It omits pagination, sorting, or limits, but for a likely small deployment list, this may be sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions already present. The description reinforces the filtering intent without adding new semantic value, thus meeting the baseline for this dimension.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List the current deployment states') and specifies the endpoint and the structure of each row. It distinguishes this tool from siblings like deploy_prompt or rollback_prompt by its listing nature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains how to filter using name/label, including the note to omit both for all entries. However, it does not explicitly mention when not to use this tool or suggest alternatives like get_deployed_prompt for a single deployment.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations, so description carries full burden. Discloses unique constraint (409 error) and Pro+ restriction. Does not mention auth needs, rate limits, or response format. Adequate but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with purpose front-loaded. No filler, but could be slightly more structured (e.g., bullet points for use cases). Efficient overall.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 6 params, no output schema, and schema covering all fields, the description is fairly complete. Covers required fields, duplicate behavior, and use case. Missing return value description is acceptable without output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage 100% with descriptions. Description adds no extra semantics beyond restating required/optional fields. Baseline score is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool registers a new prompt template (Pro+ only). It specifies required and optional fields, and mentions auto-registration for dogfood evals/experiments, distinguishing it from siblings like update_prompt and delete_prompt.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides context: 'Used when an AI agent auto-registers templates for dogfood evals / experiments' and warns about duplicate (name, version) returning 409. Does not explicitly list alternatives but context implies creating vs updating.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description must carry the full burden. It discloses that it is a PATCH endpoint (partial update) and mentions the URL-safe constraint for slug. However, it does not describe permissions, side effects, conflict behavior, or return value, leaving gaps for an AI agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is exceptionally concise with three short sentences. It front-loads the core action, then covers additional details. No fluff or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple rename tool, the description covers core functionality. However, with no output schema, it omits the return value and error conditions. It also does not address idempotency or concurrency, which are relevant for update tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds value by clarifying optionality ('Specify either or both') and the permissibility of renaming the default project, but these are minor enhancements beyond the schema's own descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action ('Update an existing project's name / slug') and specifies the resource (project). It distinguishes from siblings like create_project and delete_project by focusing on renaming, and includes a specific note about default projects.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description provides clear guidance on parameter usage ('Specify either or both') and mentions a special case ('Renaming the default project is allowed'). However, it lacks explicit when-not-to-use instructions or alternatives beyond the sibling context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses 'Partially updates' and that null clears the restriction for modelAllowlist. However, it does not mention permissions, idempotency, return value, or side effects, limiting transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences plus examples, front-loaded with purpose and scope, and contains no fluff. Every part adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers fields and gives usage examples but lacks details on return value, prerequisites, or implications of settings (e.g., enforceMode). For a mutation tool with no output schema or annotations, more context would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 33% schema description coverage, the description adds value by listing all modifiable fields and explaining the null behavior for modelAllowlist. It does not provide detailed semantics for other fields, but compensates for the low schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Update' and resource 'runtime policy gate', lists the modifiable fields, and provides examples. This distinguishes it from siblings like create, delete, and get.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions 'Pro+ only' as a licensing restriction but does not explicitly state when not to use this tool or alternatives like create_policy_gate. Examples give usage context, but no exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Given no annotations, the description carries the transparency burden. It discloses plan limitations, the special interpretation of thresholdValue for anomaly types as a sigma multiplier, and the required relationship between channelKinds and channelTargets. However, it does not mention authentication needs, rate limits, or any destructive behavior (though it's a creation).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single paragraph that efficiently conveys the purpose, main features, and an example. It avoids redundancy and is front-loaded with the core action. However, it could be slightly more structured with bullet points for multiple behavior details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (12 parameters, nested objects, multiple alert types) and no output schema, the description is incomplete. It does not cover the conditions parameter for composite alerts, nor does it mention the evalCriterionId or guardian_findings use cases. Also, it does not hint at the return value (the created alert object). While the schema covers these, the description should provide a high-level summary of all major features.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents each parameter. The description adds value by explaining the relationship between channelKinds and channelTargets with an example, the sigma multiplier range for anomaly types, and an illustrative example of usage. This goes beyond the schema's individual field descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a new alert rule, specifying the types of breaches it watches and the notification action. It distinguishes from siblings like update_alert and delete_alert by using 'create' and describing the creation process.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly compare to alternatives like propose_alert_rules or when not to use this tool. It implies usage for creating alerts but lacks usage guidance. It mentions a plan limitation (Free plan only email, up to 3 alerts) which is a constraint but not a usage alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description effectively discloses key behaviors: cascade deletion of items and preservation of past eval runs/scores. It could additionally note irreversibility but is otherwise transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences that front-load the core purpose, endpoint, and tier. Every sentence adds value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with one parameter and no output schema, the description covers the essential behavioral context (cascade, data retention). It could mention irreversibility but remains adequately complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already fully covers the single parameter (datasetId) with a description. The tool description adds no additional meaning beyond what the schema provides, so baseline score applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('delete a golden dataset'), specifies the HTTP method and endpoint, and mentions the Pro+ tier restriction. It distinguishes from sibling tools like create_eval_dataset and list_eval_datasets.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when wanting to delete a dataset but does not explicitly state when to use this tool instead of alternatives, nor does it provide exclusions or prerequisites beyond the tier.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must cover behavioral traits. It discloses key behaviors (deployment overwrite logic, per-label namespacing) and mentions the 'Pro+ only' restriction. However, it does not describe the return value, error conditions, authentication requirements, or rate limits, leaving some transparency gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—two sentences that each add significant value. The first sentence states the purpose and constraints, the second explains critical behavioral details. No superfluous words or redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given two parameters, no output schema, and moderate complexity, the description covers deployment behavior and references rollback_prompt. However, it lacks information about the response format (e.g., what is returned upon success/failure) and does not explicitly state that prompt IDs come from list_prompts (though schema says so). It is nearly complete for an action tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides 100% coverage with descriptions for both parameters ('label' with pattern and example, 'promptId' with type and source). The description adds minimal extra meaning beyond restating the deployment context and referencing 'rollback_prompt', so a baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Deploy a specific prompt version to a label'), identifies the resource ('prompt version'), and provides context ('environment such as production/staging'). It also mentions the HTTP endpoint and distinguishes from sibling 'rollback_prompt'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the behavior when a deployment already exists (prior version kept) and when re-deploying the same version (no new previous). It explicitly names 'rollback_prompt' as an alternative for reverting, but does not provide guidance on when not to use the tool or contrast with other siblings like 'get_deployed_prompt'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes what an annotation contains and clarifies independence from plaintext feature, but lacks behavioral details like pagination, ordering, or auth requirements. Without annotations, some burden is carried, but still gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences, each purposeful. Front-loaded with purpose, then defines annotations, then use case, then clarifies a nuance. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Explains return fields and usage context, but missing details on pagination, error handling, and explicit mention that it returns a list. Simple tool but could be more complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter (callId) is well-described in the schema (100% coverage). The description repeats the source ('records[].id from query_calls') which adds no new meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Return') and resource ('annotations attached to an LLM call'), distinguishing it from siblings like list_annotations_by_label (by label) and get_annotation (single annotation).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use: 'to check whether a call has human review attached or what past reviews said.' Provides context but does not explicitly mention when not to use or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses key behaviors: toggling between versions on repeated rollback, 409 errors for first deployment or deleted previous version, and Pro+ requirement. No annotation provided, so description carries full burden and does well.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences covering purpose, error conditions, and toggle behavior. No redundant information. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers behavior, errors, and toggle effect. Missing any example or mention of return value (no output schema). Fairly complete for a rollback tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema provides both parameter names and descriptions with 100% coverage. Description adds no additional meaning beyond schema. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states tool reverts a deployed prompt to previous version, specifying verb (revert), resource (prompt deployed to label), and adds detail about versioning and toggle behavior. Distinguishes from siblings like deploy_prompt.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implicitly clear when to use (to undo a deployment), but lacks explicit guidance on when not to use or comparison to alternatives. No exclusions or alternatives mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the behavioral nuance that groupBy='none' still returns per-provider breakdown with overall sum in response.total, but it does not mention whether the tool is read-only, any authentication requirements, or potential side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two concise sentences with no wasted words. The first sentence clearly states the primary purpose, and the second provides a critical behavioral note. It is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has only two enum parameters and is fairly simple, but the description omits details about the response structure (besides mentioning response.total) and any error conditions. Given no output schema, a more complete description would include typical return fields or constraints.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of parameters with descriptions, setting a baseline of 3. The description adds extra meaning beyond the schema by explaining the backend compatibility behavior when groupBy='none' and pointing to the response.total field, which is not in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Return') and resource ('cost / call count / token aggregates per time range') and distinguishes itself from siblings like get_llm_budget or get_percentiles by clearly stating it provides aggregates with per-provider breakdown.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving cost summaries, and the note about groupBy='none' clarifies a special case. However, it does not explicitly state when to use this tool over alternatives such as get_llm_budget or get_percentiles, though the purpose is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden. It states 'Partially updates' which implies it is not destructive, but does not disclose any other behavioral traits such as auth requirements (Pro+ only is a plan restriction, not direct auth), rate limits, or idempotency. Somewhat adequate but could be more transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise: two sentences plus an example list. Every sentence adds value, and the information is front-loaded. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (4 parameters, 1 required, no output schema), the description adequately covers what the tool does and how to use it with concrete examples. It is complete for an agent to understand and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with each parameter having a description. The tool description adds value by providing usage examples mapping to parameters (e.g., 'raise the limit to $100' to monthlyLimitUsd). However, it does not substantially enhance understanding beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the action 'Update' and the resource 'runtime budget gate'. It mentions 'Partially updates any of monthlyLimitUsd / enforceMode / enabled', which distinguishes it from sibling tools like create, delete, or get budget gates.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Examples are provided ('raise the limit to $100', 'disable the gate temporarily', 'switch to fail_closed') that illustrate typical use cases. However, there is no explicit guidance on when not to use this tool or alternatives, such as when to use create_budget_gate instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description discloses default type/scope, error conditions, and deterministic evaluator behavior (free/instant). It does not detail rate limits or auth but covers key behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Multiple sentences, some verbose (e.g., final sentence about AI agent path). Core purpose is front-loaded but could be tightened. Still organized and informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 8 params, no output schema, and nested config, the description covers required fields, constraints, types, defaults, error behavior, and deterministic evaluator details. Missing return format but otherwise complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, baseline 3. Description adds context: deterministic types require config, rubric needed even for deterministic, examples for config fields, and default type/scope.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create one custom eval criterion' and specifies the scope 'in your account (Pro+ only)'. It distinguishes from siblings like get_eval_criterion, list, update, delete by focusing on creation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It mentions required fields, uniqueness constraint causing 409, and when to use deterministic types vs llm_judge. However, it does not explicitly contrast with propose_eval_criteria or other creation tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the critical behavior of overwriting when the name exists ('overwrite when the name exists'), which is a key trait for an upsert operation. It also mentions name uniqueness and filter structure. No contradictions exist because no annotations are present.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with three sentences. The first sentence states the main action and overwrite behavior. The second details the filter shape, and the third provides an example. No extraneous information; every sentence serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has two parameters (one nested), no output schema, and moderate complexity. The description adequately covers the main purpose and parameter meaning. However, it does not mention the return value of the tool, which would be helpful for an agent to know what to expect upon creation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds some value by explaining the naming uniqueness and providing an example use case ('last 7 days, GPT-4 only'), but it does not add significant parameter-level detail beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the tool's action: 'Create a new saved view, or overwrite when the name exists'. It specifies the resource (saved view) and the verb (create/overwrite), and the example ('last 7 days, GPT-4 only') clarifies the purpose. Among siblings like list_saved_views and delete_saved_view, this tool is clearly distinct.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool: 'Lets an AI agent save frequently used filters under a name'. It provides context for use and mentions name uniqueness and overwrite behavior. However, it does not explicitly state when not to use or compare with alternatives, though the sibling list provides enough differentiation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden. It implies a read operation (returning data) but does not explicitly state it is read-only, nor does it mention authorization needs, rate limits, or other side effects. Adequate but could be more transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two sentences, front-loading the core purpose in the first sentence and providing usage guidance in the second. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a single parameter and no output schema, the description adequately explains what is returned (configuration and history) and how to get the ID. It covers the essential information for an agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the alertId parameter. The description adds value by specifying the origin of the ID ('from list_alerts') and listing what to check, which goes beyond the schema's basic 'Target alert ID'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns the detailed configuration and recent trigger history of an alert. It lists specific attributes (threshold, notification channels, silence state, when it fired), distinguishing it from sibling tools like list_alerts or list_alert_events.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells the agent to pass the alertId obtained from list_alerts and to use it for checking specific details. While it doesn't state when not to use it, the context among siblings (create, update, delete) makes the use case clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes return content (detail plus all items) and idempotent fetch behavior. No annotations provided, so description carries full burden. Could mention that it is read-only.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with verb, no extraneous words. Efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequately describes purpose, endpoint, and parameter source for a simple fetch tool. Lacks details on response format, but no output schema exists. Could mention potential limitations like pagination.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers parameter with name, type, and basic description. Description adds value by specifying the source (list_eval_datasets.datasets[].id), aiding correct invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action (Fetch), resource (one dataset's detail plus all items), and explicit endpoint (GET /v1/eval-datasets/:id). It distinguishes from siblings like list_eval_datasets and delete_eval_dataset.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not-to-use guidance. Implicitly indicates dataset ID comes from list_eval_datasets, but does not compare with siblings like get_eval_run or get_eval_criterion.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description carries full burden. It discloses ordering, output fields (scoredCount, failedCount, meanScoreByCriterion), scope to account, and free user access.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with purpose, then ordering and output fields, then access note. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Completes the picture for a listing tool: tells what output includes, ordering, scope. No missing critical info given single parameter and no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear description for the limit parameter. Tool description adds nothing beyond schema, so baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists eval run history, scoped to account, most recent first. It distinguishes from siblings like list_eval_datasets and compare_eval_runs by explicitly mentioning the summary fields.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use: to get recent eval summaries and trends. However, it does not explicitly exclude alternatives or mention when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must cover behavioral traits. It discloses read-only access and scope, but does not detail auth needs, side effects, or output format. Adequate but not thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose. Every sentence adds value, with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with no parameters and no output schema, the description explains what it returns and what it cannot do. It is sufficient, though missing details on sorting or pagination.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so schema coverage is 100%. The description adds no parameter info because none exist, which is appropriate. Baseline for zero params is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns unresolved improvement proposals from the Argosvix guardian, listing specific categories. It uses a specific verb and resource, distinguishing it from siblings like reply_proposal.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states that approvals, dismissals, and executions are handled in the dashboard inbox, and agents can only read and converse. This sets clear boundaries, but does not name alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, but description discloses the max limit of 20 per account and indicates it's a list (read) operation. Could be more explicit about read-only behavior, but sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise two sentences: first sentence states the action, second explains the concept and constraint. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless list tool with no output schema, the description provides enough context: what it returns, the max limit, and the concept. Could include expected return format, but not critical.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist; schema coverage is 100%. Description adds context about saved views, which adds value beyond the empty schema. Baseline for 0 params is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'List' and the resource 'saved views'. Explains what a saved view is with an example, and differentiates from siblings like create_saved_view and delete_saved_view.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implicitly suggests use when wanting to see existing saved views, but lacks explicit when-to-use vs alternatives or when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry the burden. It states the effect (stops notification delivery) and default duration. However, it does not disclose return behavior, idempotency, or reversibility (though unsilence_alert exists). Adequate but could be more explicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences: purpose, parameter details, instruction. It is concise and front-loaded with no redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity (2 parameters, no output schema), the description covers essential behaviors. It lacks mention of the return value but is otherwise complete for a straightforward mute operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, providing a baseline of 3. The description adds value by explaining the default for 'until' and instructing to get alertId from list_alerts, which provides context beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Temporarily mute an alert (stops notification delivery)' using a specific verb and resource. It distinguishes from siblings like unsilence_alert and acknowledge_alert.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description tells when to use the tool by mentioning the default 24-hour expiry and custom until timestamp. It also instructs to pass alertId from list_alerts, implying a prerequisite. However, it does not explicitly mention when not to use or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description carries full burden. It discloses partial update behavior, effect of null secret (removes signature), re-enabling resets failure counter, and 404 for other accounts' webhooks. Lacks details on idempotency or rate limits, but covers key behavioral traits well.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is three sentences, front-loaded with core action and HTTP verb, and includes key behavioral notes without unnecessary words. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 6 parameters and no output schema, description explains partial update, special behaviors, ownership, and id source. Lacks mention of response format, but overall sufficient for agent to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all parameters (100% coverage), so baseline is 3. Description adds value by clarifying that only specified fields change, and provides special behaviors for 'secret' and 'enabled'. This goes beyond the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it partially updates an outbound event webhook, specifies the HTTP method and endpoint, lists the editable fields, and distinguishes from create/delete/tool siblings. The purpose is specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description explains when to use (update webhook) and ownership restriction, but does not explicitly guide when NOT to use it or mention alternatives like test_webhook or retry_failed_webhook. Implicitly suggests using list_webhooks first to get the id.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses return format (deltas, verdict), verdict options, failed count definition (scores <= 2), and account constraint. With no annotations, this covers key behavioral traits adequately.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single paragraph with clear progression: action, value, details. Efficient but could be slightly more concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Explains key outputs (deltas, verdict) and failed count rule. No output schema, so description carries load. Adequately complete for a comparison tool, though could mention edge cases like identical runs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds context beyond schema: baselineRunId references list_eval_runs.runs[].id, candidateRunId must be from same source and different from baseline. Schema coverage is 100%, so description adds meaningful usage guidance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it compares two eval runs, returning per-criterion mean score deltas, failed count delta, and a verdict. Distinguishes from siblings like list_eval_runs and get_eval_run by focusing on comparison.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Describes use for measuring prompt improvement and regression detection. Implicitly contrasts with single-run access tools, but does not explicitly exclude cases where alternative tools are more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description discloses key behaviors: the HTTP DELETE method, no cascading dependencies, and a safety recommendation. It does not mention irreversibility or auth requirements, but for a simple delete operation this is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no redundant information. Every sentence adds value: the first states the action, the second provides safety context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has one parameter and no output schema. The description fully covers the purpose, behavior, and a usage recommendation, making it complete for its simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description does not add extra meaning beyond the schema's description of annotationId as an AUTOINCREMENT integer. No parameter-level details are missing.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Delete an annotation' with the HTTP method and resource identifier. It distinguishes from sibling tools like create_annotation and update_annotation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states no CASCADE impact and recommends verifying with get_annotation before deletion. Provides clear context for safe usage, though does not explicitly mention when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses account-scoping and 404 behavior, but does not explicitly state read-only nature, idempotency, or permissions. Adequate but could be more thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no wasted words. Front-loaded purpose, then usage and behavioral details. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, return contents (template, variables, labels, description), id provenance, scoping, and error behavior. For a simple get-by-id tool without output schema, this is fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by instructing to use prompts[].id from list_prompts as-is, which is beyond the schema's description. This provides practical context for the parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Fetch one prompt's detail by id,' specifying the verb, resource, and method. It distinguishes from siblings like list_prompts (multiple) and get_deployed_prompt (deployed version) by mentioning the id usage and what is included.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly advises using prompts[].id from list_prompts, providing a direct link to the source of the id. It also notes account scoping and 404 behavior for invalid ids, guiding proper usage. Lacks explicit mention of alternatives like get_deployed_prompt but context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; the description implies a read-only operation by stating 'return', but does not explicitly confirm safety, permissions, or error handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, both essential: first defines purpose, second gives usage guidance. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and no output schema, the description fully covers what the agent needs to know.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a clear description of proposalId. The description adds value by specifying the source (list_proposals) and format (starts with prp_).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states it returns the thread for a proposal (questions and AI replies), differentiating from sibling tools like list_proposals or reply_proposal.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description tells the user to get proposalId from list_proposals, providing a clear prerequisite. It does not explicitly exclude other uses, but the context is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description carries the full burden. It reveals the HTTP method (GET), that archived projects are excluded, and plan restrictions, making behavior fairly transparent for a read-only list.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences that front-load the core purpose and add additional context without redundancy. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Lacks output format details (e.g., response fields). While adequate for a simple list, providing expected return fields would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters in schema, so baseline is 4. The description adds relevant context about per-environment filtering and plan limitations, which is useful beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists active projects, excluding archived, and specifies the API endpoint. It distinguishes from other project tools like create_project and delete_project.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides context for per-environment observation and plan limitations, but does not explicitly state when not to use alternatives like list_prompts. Overall clear usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations exist, the description carries full burden. It discloses the range ($5-$500), increments, carry-over of existing spend, monthly resets, and behavior when setting below current spend. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: purpose first, then details, then examples. Every sentence adds value with no redundancy. Slightly longer but appropriate for the content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter tool with no output schema, the description covers purpose, constraints, behavioral nuances, and usage examples. It is sufficient for an AI agent to select and invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a description for budgetUsd. The description adds context like 'hard cap against runaway spend', carry-over behavior, and concrete examples, which enrich understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action ('Raise or lower') and the resource ('monthly LLM feature budget'). It distinguishes from siblings like 'get_llm_budget' by specifying it modifies the budget, and the 'Pro+ only' constraint adds specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear usage context with examples ('we hit 80% of the budget') and mentions the 'Pro+ only' eligibility. It lacks explicit when-not-to-use but the examples guide appropriate scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses that it's Pro+ only, uses POST, changes both name and version, and can cause 409 collision. However, it does not describe the response format or any side effects (e.g., on deployments). Still, key behavioral traits are covered.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences long, with each sentence adding value: purpose, use case, and differentiation. It is well-structured and front-loaded but could be slightly more concise (e.g., merging the first two sentences).
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema, the description covers licensing, collision behavior, and distinguishes from related tools. It could mention what the API returns (e.g., the renamed prompt) but is otherwise sufficient for a straightforward rename operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all three parameters. The description adds context about typo fixes and collision but does not add parameter-specific semantics beyond what the schema already provides. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Change' and resource 'existing prompt's name + version', and distinguishes from update_prompt by noting that update_prompt never changes name/version. The example and explicit reference to POST /v1/prompts/:id/rename provide specific purpose clarity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Main use is typo fixes' and contrasts with update_prompt, explaining that rename is a separate tool for semantic separation. It also mentions collision with existing (name, version) results in 409 error, providing clear when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so description carries full burden. Discloses mutating nature (POST), cost (~$0.01/run), scoring algorithm, and return behavior (summary with scoredCount and reason). Lacks mention of idempotency handling or full response structure, but adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single paragraph, dense but efficient. Front-loaded with the main action and endpoint. Could benefit from bullet points for preconditions, but still clear and concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 5 parameters, no output schema, and important preconditions, the description covers almost everything: failure modes, costing, preconditions, and partial return structure. Missing full response shape but sufficient for agent to invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with good descriptions. The tool description adds context linking parameters to behavior (e.g., recentCount defines how many calls, label as substring filter, idempotencyKey for dedup). Adds value beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb (Start a new eval run immediately), resource (eval-runs via POST), and what it does (scores recent calls against default and custom criteria). Distinct from siblings like compare_eval_runs or run_eval_dataset.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit conditions: Pro+ only, needs OPENAI_API_KEY, requires plaintext storage opt-in. Gives failure modes (403, 500, zero scored). Does not explicitly contrast with run_eval_dataset, but context is strong.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full burden. It discloses full replace behavior, required fields, consequences of omitting type/config (revert to defaults), and necessary config for deterministic types. It also covers error scenarios, providing good transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured paragraph that front-loads the main action. It is concise, covering key details without wordiness. While it could be broken into bullet points, it remains efficient and informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (9 params, nested objects, no output schema), the description is remarkably complete. It explains the full replace nature, required fields, default behaviors, and error conditions. This is sufficient for an agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value beyond the schema by explaining the full replace semantics, required vs optional fields, and how type/config are overwritten. It also clarifies the impact of omissions and error conditions, enhancing parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool updates a custom criterion with a full replace, specifying it is Pro+ only and uses PATCH /v1/eval-criteria/:id. It differentiates from siblings like create_eval_criterion and delete_eval_criterion by emphasizing the full replace behavior and the requirement of all fields.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use: for full updates of custom criteria, not partial. It also gives error conditions (404 for out-of-scope criteria, 409 for name collisions). However, it does not explicitly state when not to use or compare to alternative tools, but the context is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that frozen=true makes items permanently immutable, which is a critical behavioral detail. It also mentions the item limit. However, it does not describe the return value or error conditions, missing some behavioral details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences: first defines the action, second covers limits, third explains frozen. All sentences are essential and front-loaded. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers key constraints and behavioral aspects. Lacks return value description and prerequisites (e.g., authentication), but for a creation tool with no output schema, it is mostly complete. The sibling tools provide context for retrieval.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value beyond the schema by explaining that inputText is fed to the model and expectedOutput is used as a judge reference, and that name must be unique within the account. This enriches parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action 'Create a golden dataset' with the HTTP method and resource, and includes the 'Pro+ only' restriction, distinguishing it from other dataset-related tools like run_eval_dataset or get_eval_dataset.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides important usage constraints: Pro+ only, up to 20 test cases, up to 50 datasets per account, and the effect of frozen=true. However, it does not explicitly mention when to use this tool versus alternatives or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Although no annotations exist, the description adds the Pro+ edition restriction. The destructive nature is implied by 'delete', but additional details on permissions or reversibility would improve transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise with just two sentences, no unnecessary words, and perfectly front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with one parameter and no output schema, the description covers the action, edition requirement, and an alternative. Could mention permissions or consequences, but overall adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter, and the description does not add new information about the parameter beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it deletes a runtime policy gate and specifies the Pro+ edition requirement. It distinguishes from update_policy_gate by suggesting an alternative for temporary pausing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly recommends using update_policy_gate with enabled: false for temporary pausing instead of deleting, providing clear when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It clearly explains the soft-delete mechanism (`archived_at`), the restriction on the default project, and the impact on related data (calls/alerts remain). This is transparent for a deletion tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise, consisting of two sentences with no unnecessary words. It front-loads the key action and endpoint, then covers important details efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema, the description explains the consequences of deletion (soft-delete, data retention, next steps). It is mostly complete but could mention undoability or more detailed consequences.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one parameter with a description that matches the tool's description. The tool-level description adds behavioral context about the parameter (e.g., the default project cannot be deleted), providing extra meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool performs a soft-delete on a project, specifies the HTTP endpoint, and distinguishes itself from other delete tools by focusing on projects. The caveat about the default project being undeletable adds specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implicitly indicates when to use the tool (to delete a project) and provides important context about limitations (default project cannot be deleted) and post-deletion behavior. However, it does not explicitly state when to avoid using it or suggest alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Given no annotations, the description carries full burden. It discloses the destructive nature, the 404 error for unauthorized webhooks, and the parameter source. However, it does not explicitly state irreversibility or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no extraneous words. All information is front-loaded and essential.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with one parameter and no output schema, the description covers the action, endpoint, plan requirement, error case, and parameter source. It is nearly complete, though could mention finality.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by specifying that webhookId comes from list_webhooks, which aids in parameter population beyond the schema's basic description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it deletes an outbound event webhook, specifies the HTTP method and endpoint, and differentiates from sibling tools like create, list, update, test, and retry by focusing on deletion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides usage context by noting it's Pro+ only and that other accounts' webhooks return 404, but does not explicitly compare with alternatives or state when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully handles behavioral disclosure. It explains the export operation, plan-specific limits, limit clamping behavior, and that the result format matches query_calls. This provides adequate transparency without contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise—three sentences with no wasted words. It front-loads the purpose and key differentiator, then adds necessary details (limits, filters, example). Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters, 100% schema coverage, no output schema, and sibling tools like query_calls, the description is sufficiently complete. It mentions the result format and provides guidance on limits and filters, covering essential context for correct usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (all parameters described). The description adds value by explaining the 'limit' parameter's clamping behavior and summarizing filter axes, which goes beyond the schema's individual descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Large-batch export of calls (POST /v1/query/export).' It distinguishes from the sibling query_calls by highlighting a higher limit and per-plan maximums, providing a specific verb+resource+scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly compares to query_calls, noting 'Higher limit than query_calls' and per-plan record limits. It provides filtering axes and an example usage, offering clear context for when to use this tool instead of alternatives. No explicit when-not-to-use, but the comparison suffices.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavior: it is a pure read aggregator that fetches endpoints in parallel, returns partial results if one endpoint fails, and defines clear thresholds for 'critical' and 'warn' states. This exceeds expectations for transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is detailed but well-structured, starting with purpose, then return type, thresholds, example, and behavior. It could be slightly more concise, but every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Without an output schema, the description fully explains the return structure (including exact field names and types) and behavior on partial failures. This is complete for the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter 'window', and the schema already includes a description. The tool description does not add additional meaning beyond the schema, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it gets a health summary of LLM infrastructure by fetching 4 existing endpoints in parallel, distinguishing it from sibling tools like aggregate_calls, get_percentiles, etc. The verb 'get' and resource 'account health' are specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides an example query ('how is our LLM infra doing right now?') and explains the tool returns a compressed summary, implying it should be used for quick health checks. However, it lacks explicit when-not-to-use guidance or direct comparisons to alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Describes output structure (scores array with integer and reasoning) and references endpoint. Lacks details on error handling or existence checks, but adequate for read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each with distinct purpose: action, input sourcing, output details. Front-loaded and no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, but description details key parts of return value (scores array structure). Lacks description of 'detail' object, but sufficient for a fetch tool with clear input pattern.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter (runId) with full schema coverage. Description adds value by instructing to use ID from list_eval_runs 'as-is', providing semantic context beyond the schema's 'Target eval run id'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'Fetch' and resource 'eval run's detail plus scores'. It distinguishes from sibling 'list_eval_runs' by instructing to use its ID.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance to use 'runs[].id from list_eval_runs as-is', indicating when to use this tool after listing. Does not mention when not to use or alternatives like compare_eval_runs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It explains the computation method (nearest-rank via window functions) and notes implementation specifics (D1 SQLite limitation). It also clarifies output behavior (single value vs. time series). Some details about response format are missing, but the key behaviors are disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences convey the core purpose, available options, and implementation detail. No wasted words; every sentence adds meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 6 optional parameters and no output schema, the description covers essential behavior (metric types, time grouping, computation). It lacks explicit mention of return structure, but the example phrasing and method hint at the output. Overall, it is sufficiently complete for an agent to understand the tool's function.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds value by explaining the metric and groupBy semantics in natural language, including defaults and example. It goes beyond the schema by describing how to interpret groupBy (time bucketing) and provides a concrete usage example.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves percentile metrics over calls, specifies metric options (latency/cost) and time series capability with groupBy. It references the specific API endpoint and provides an example phrase, making the purpose distinct from sibling tools like query_calls or get_cost_summary.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates when to use the tool (for percentile queries) and describes the groupBy options for time series. It does not explicitly state when not to use it or compare to alternatives, but the context is clear enough for an agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It explains the response format (policy object or null), mentions SDK local evaluation, and gives behavioral context. It lacks disclosure of authentication needs or rate limits, but for a zero-parameter read operation this is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized with each sentence adding value: purpose, response shape, SDK context, example queries. It could be slightly more concise but is well-structured and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and no output schema, the description fully explains the return value and usage context. It includes example phrasings and mentions the relationship to SDK configuration, making it complete for a simple getter tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so baseline is 4. The description adds no parameter information, but none is needed since schema coverage is 100% (vacuously).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Get the runtime policy gate settings', using a specific verb (get) and resource (policy gate). It distinguishes itself from sibling tools like create_policy_gate, update_policy_gate, and delete_policy_gate by focusing on reading configuration.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides example phrasing ('what model restrictions are active right now?', 'is PII blocking enabled?') that tells the agent when to use this tool. It mentions runtime control plane context but does not explicitly state when not to use it or list alternatives, though the sibling list implies other actions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full behavioral burden. It discloses sorting order, account-wide scope, and maximum return count, plus label format constraints. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two efficient sentences: first states core function, second adds examples and format constraints. No wasted words, information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers all essential aspects: what it returns (annotations by label), ordering, scope, limit. Could mention pagination or response structure but not critical for a simple list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers both parameters fully. Description adds value via usage examples for the label parameter, clarifying its intent beyond the schema's technical description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns annotations by label, with scope 'account-wide' and order 'newest first', distinguishing it from sibling list_annotations_for_call which is per-call.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides concrete examples ('collecting calls rated good', 'listing human reviews labeled bug') that clarify when to use. Lacks explicit exclusion of alternatives, but context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description fully bears the burden. Discloses source types, behavior with/without callId, case of missing API key, precondition, and empty array semantics. Clearly read-only.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Relatively verbose but each sentence adds unique information. Front-loaded with core action. Could be slightly trimmed but effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, so description should explain return structure. Mentions ordering and empty array meaning, but does not describe fields of each assessment (e.g., content, severity). Gap in completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but description adds meaning: ordering when no callId (flagged first then most recent), clarifies callId targets llm_calls.id, and provides context like default/max already in schema. Adds value beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it lists safety classifier assessments (OpenAI Moderation), specifies filtering by callId or account-wide, and distinguishes from sibling tool classify_calls_batch.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explains when to use with/without callId, precondition that safety classification is disabled by default, meaning of empty array, and use case for AI agents. Does not explicitly list when not to use, but contrasts with classify_calls_batch.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the burden of behavioral context. It discloses that the update is partial (PATCH), callId is immutable, and the annotationId must come from list_annotations_for_call. This is valuable beyond the schema, though it doesn't cover idempotency or permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no redundant information. It front-loads the action and specific fields, then provides examples and sourcing instructions. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the tool's purpose, parameters, and usage context well. It is sufficient for a simple partial update tool with 4 parameters and no output schema. Missing explicit error handling or permissions, but not critical for basic usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds significant value by explaining how to source annotationId and reaffirming that omitted fields retain current values. This goes beyond the schema's parameter descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool performs a partial update on an annotation's annotationText, label, or qualityScore, using the PATCH method. It distinguishes from sibling tools like create_annotation or delete_annotation by specifying 'update' and giving examples like 'fixing a label or re-scoring quality'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides usage context by instructing to obtain annotationId from list_annotations_for_call and noting that callId is immutable. It gives examples of when to use the tool (fixing label, rescoring quality). However, it does not explicitly exclude alternative tools (e.g., delete_annotation) beyond the sibling list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description fully discloses behavioral traits: limits on hour/minute modes (168h, 60min), required tagKey pattern, error mode behavior, and exact formulas for each metric. It also describes the return format. Since no annotations are present, this burden is met excellently.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise yet comprehensive: it opens with the core purpose, then systematically covers dimensions, special modes, metric definitions, and output format. Every sentence adds value, and the structure is front-loaded and logical.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 parameters, no output schema, no annotations), the description is remarkably complete. It covers purpose, usage example, constraints, metric semantics, and output shape. Only missing explicit sibling comparisons, but these are covered by purpose clarity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds significant value beyond schema by explaining metric formulas (e.g., cost = SUM(cost_usd)), caps for hour/minute, tagKey requirements, and return structure. This exceeds the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool computes an aggregation cube over calls, specifying groupBy dimensions and metrics with concrete examples. It distinguishes itself from siblings like query_calls or get_cost_summary by focusing on multi-dimensional aggregation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
While the description provides a usage example ('aggregate this month's cost by model'), it does not explicitly state when to use this tool versus alternatives such as query_calls, get_cost_summary, or get_percentiles. The guidance is implied but not precise.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses the empty-annotation error condition but could mention success response structure or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Compact paragraph with front-loaded purpose. All sentences add value, though could be slightly more structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, usage, constraints, and tool synergy (run_eval). Lacks return value description but output schema absent makes this acceptable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 100% coverage, but description adds critical constraint ('at least one of') and clarifies label pattern and lengths, exceeding schema baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Create a new annotation' with specific resource and context (LLM call). Examples distinguish from sibling tools like update_annotation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states required combination of optional fields to avoid 400 error. Provides concrete examples and mentions integration with eval pipeline.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that redact mode is not supported, SDK blocks calls before execution, and at least one rule is required. No annotations provided, so description carries full burden and does so well.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Messages are front-loaded and information-dense, but includes a longer example phrasing. Could be slightly trimmed without losing clarity, but overall efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, description explains constraints, account limits, and SDK interaction. Does not mention return value, but likely expected for a creation tool. Adequately complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, baseline 3. Description adds value by summarizing parameter purposes (modelAllowlist, blockPii, blockSecrets), gives examples, and explains PII coverage details beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states verb 'Create' and resource 'runtime policy gate', describes its function (model allowlist/PII/secret block), and distinguishes from sibling tools like delete/update/get policy gate. Includes example phrasing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Specifies Pro+ only, required rules (at least one), account limit (one), and SDK opt-in. Does not explicitly state when not to use, but constraints are clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that deletion disables SDK pre-execution enforcement and that it is Pro+ only. Does not contradict annotations (none provided). Lacks details on reversibility or other side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded, no unnecessary words. Every sentence provides value: action, consequence, and alternative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With only one parameter and no output schema, the description is complete. It explains the tool's effect, the only parameter, and provides usage context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a description of gateId. The description adds no new information beyond the schema, so baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete a runtime budget gate' with a specific verb and resource. It distinguishes from sibling tool 'update_budget_gate' by mentioning an alternative for temporary pausing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance: 'To pause temporarily, prefer update_budget_gate with enabled: false', indicating when not to use delete and offering an alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full behavioral disclosure. It mentions the structural defense (404 for other accounts) and lists returned fields. It could add more details like authentication requirements or error states, but it covers the key behaviors for a read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no extra words. First sentence gives purpose and id source, second lists data and added behavior. Information is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple GET by ID tool with one parameter and no output schema, the description is complete. It mentions the source of the id, the fields returned, and a security behavior. No missing critical information for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by stating that the id comes from list_annotations_*, providing context beyond the schema's 'Target annotation id' description. This helps the agent understand the correct source for the id.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Fetch one annotation by id', specifying the verb and resource. It distinguishes from sibling tools like list_annotations_* that retrieve multiple, and create/update/delete tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description specifies that the id should be obtained from list_annotations_*, guiding the agent on prerequisite steps. It also notes that ids from other accounts return 404, a usage constraint. However, it does not explicitly exclude use cases like updates or deletions, but the context from sibling names makes that clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses the returned fields (gates, spentUsdThisMonth, monthStart, ttlSeconds) and notes that monthStart is UTC. It also mentions the tool's relationship to the SDK's budgetGate opt-in. Lacks details on freshness/caching or authorization, but is fairly transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concisely structured: first sentence summarizes purpose, then response format, then contextual details, then differentiation, then example queries. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description sufficiently explains the response fields and behavior. It covers the core functionality and distinguishes from related tools. Could potentially mention if multiple gates are expected, but overall it is complete for a straightforward retrieval tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters, and the description appropriately indicates no inputs are needed. Schema coverage is 100% (empty schema), so no additional parameter details are required.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves budget gate settings and monthly LLM spend. It specifies the response structure and explicitly distinguishes from the sibling get_llm_budget tool, making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use this tool ('how much budget gate headroom is left this month?') and distinguishes it from get_llm_budget. It could be improved by explicitly stating scenarios where it should not be used, but the distinction and examples are helpful.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses HTTP method, endpoint, availability (Free), return fields (template/variables/labels/version), and error case (404 when nothing deployed). No annotations to contradict.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with main action, no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read operation with 2 parameters and no output schema, the description covers key aspects: purpose, return fields, error condition, and endpoint info.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both name and label parameters. Description adds minimal value by grouping them as 'environment (name + label)', but does not provide additional details beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'Resolve and return' and resource 'prompt version currently deployed', and distinguishes from siblings like get_prompt by specifying 'deployed' and 'main runtime path for an agent to fetch the production prompt'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides context 'main runtime path for an agent to fetch the production prompt' implying when to use, but does not explicitly state when not to use or directly contrast with alternative tools like get_prompt.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses admin scoping, cursor pagination format (nextCursor = 'created_at|id'), max limit 200, and available filters. Without annotations, it covers essential behavioral traits well. Could add that it is read-only, but 'list' implies that.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences, front-loaded with purpose and scoping. No redundant information; every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description still explains pagination format and max limit. Parameters are fully covered in schema and description. For a list tool with 0 required params and 7 optional, it is complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds examples for eventType, explains cursor format, and clarifies scope. This provides meaningful additional context beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it lists the audit log, scoped to account, and specifies the REST endpoint. No sibling tool duplicates this functionality, so it's well-distinguished.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly mentions admin role requirement and that viewer/member get 403, giving clear context. Does not explicitly state when not to use or provide alternatives, but usage is well-understood.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full behavioral burden. It indicates this is a read-only GET operation and describes the response fields. While it does not mention auth or rate limits, the simplicity of the operation makes this adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: first gives purpose and endpoint, second details return fields and context. No extraneous content; every sentence serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description sufficiently explains the return value structure and the semantic meaning of golden datasets. It is complete for a parameterless list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, and schema coverage is 100%. The description adds value by explaining the returned data (name, description, item count, frozen state) and the concept of a golden dataset, so it exceeds the baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists golden datasets using a GET endpoint, specifies the returned fields (name, description, item count, frozen state), and provides context explaining what a golden dataset is. This differentiates it from sibling tools like list_eval_runs or list_eval_criteria.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implicitly defines when to use this tool (to view all golden datasets) and hints at related actions (run_eval_dataset). However, it lacks explicit when-not or alternative usage instructions, which would elevate it to a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavior: read-only, HTTP method and path, return fields (email, role, status, joined-at), and exclusion of removed members. No hidden side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences: first states purpose and endpoint, second details return fields, third clarifies excluded operations. Front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Reasonable completeness given no parameters and no output schema: describes return fields and scope. Lacks mention of pagination or limits, but acceptable for a simple list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so baseline 4 applies. Description adds value by explaining tool scope and return data, though no parameter details are needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it lists a Team account's members, specifies the API endpoint, and excludes removed members. Differentiates from siblings by focusing on members and mentioning exclusive operations, though not explicitly contrasting with other list tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states the tool is read-only and that invitations, role changes, and removals are not exposed via MCP, directing users to the dashboard. This provides clear when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description carries the full burden. Describes listing behavior, returned fields, filtering, limit, and sort order. Implicitly read-only (listing templates), but does not explicitly state no side effects or permission requirements. Still, the behavior is well-covered for a list operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single paragraph with 3 well-structured sentences. Front-loaded with purpose, then filtering details, then limit/sort info. Every sentence adds necessary information; no redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a list tool with 3 optional params, no output schema, and no annotations, the description covers all essential aspects: purpose, filters, limit, sort order, returned fields, and its role in the dashboard. No critical gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (all 3 parameters documented). Description adds value by explaining how to use filters (e.g., label example 'production'), that name fetches all versions, and adds context about sort order and limit. This goes beyond the schema's basic descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool lists prompt templates, enumerates included fields (id/name/version/template/variables/labels/description/createdAt), and notes it is the main path for an AI agent to read prompts, distinguishing it from sibling tools like get_prompt or create_prompt.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear usage context: filtering by label or name, with example values and exact match semantics. Also specifies limit (up to 200) and sort order (name ASC + created_at DESC). Does not explicitly exclude scenarios or mention alternatives, but gives sufficient guidance for typical use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description carries the full burden. It discloses that the secret is never returned and that webhooks use signed POSTs. It does not mention pagination or rate limits, but for a parameterless read operation, the transparency is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (two sentences) and front-loaded with the main action. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and no output schema, the description fully explains the tool's purpose, return fields, and the broader context of webhooks. It is complete for an agent to decide when to call it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, and schema coverage is 100%. The description adds context about the return fields, which is useful but not required for a no-parameter tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists registered webhooks, specifies the HTTP method and path, and enumerates returned fields. It distinguishes itself from sibling tools like create_webhook, delete_webhook, and update_webhook.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly states when to use the tool (to list webhooks) and notes availability on Free tier. No explicit when-not-to-use or alternatives are provided, but it's clear from context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully discloses behavioral traits: it is a mutation that updates alerts.silenced_until and emits audit events, details dryRun vs. actual execution, idempotency mechanism, per-account scoping, and the absence of a founder gate for paid users. This is comprehensive and adds significant context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is relatively long but efficiently structured with clear sections for purpose, parameters, behavior, and output. Every sentence adds value, though some minor redundancy exists (e.g., repeating endpoint details). It is front-loaded with the core use case.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 parameters, no output schema), the description is very complete: it explains both dryRun and actual outputs, idempotency, permissions, and relationships with sibling tools. Minor omissions include explicit error conditions or rate limits, but overall it provides sufficient context for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers all 6 parameters with descriptions (100% coverage), but the description adds important context beyond the schema: it explains the mutual exclusivity of alertId and byVolumeThreshold, default and range for silenceDurationMinutes, optional reason, dryRun behavior, and approvalId usage. It also describes the idempotency mechanism, which is not in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool is for bulk-silencing noisy alerts that fired repeatedly in the past hour, using specific verbs and resources. It distinguishes itself from sibling tools like silence_alert and unsilence_alert by emphasizing the automated, volume-based triggering and the reversal capability.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit guidance on when to use the tool (e.g., finishing the prompt about repeated alerts) and how to choose between alertId and byVolumeThreshold. It mentions that unsilence_alert can undo the action and that Pro+ users can call it directly, but it could be improved by explicitly stating when not to use it (e.g., for single alert without volume criteria).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description must fully disclose behavior. It details the backend enforcement (plan gate, budget gate), limitations (maxRecords with 1-100 range and default 50), return format (scanned, assessed, flagged, failures, skipped), data lineage (source='mcp'), and audit event emission. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is moderately concise, with each sentence providing distinct value. It is front-loaded with the core purpose. A minor redundancy exists with the parameter default mentioned both in the description and schema, but overall well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description thoroughly explains the return values and side effects (audit event, source tracking). With only one parameter and clear usage context, it covers all necessary information for an agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter maxRecords, with a detailed description including min/max/default and technical notes. The main description restates the default value but adds no significant new semantics beyond the schema. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool performs batch safety classification of unclassified calls on demand, specifying the verb 'classify', resource 'unclassified calls', and method 'batch on demand'. It distinguishes itself from the cron job by offering a real-time path, making the purpose unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use this tool ('right now' path for immediate classification) versus the cron job (every 15 min). Notes Pro+ plan requirement and that Free plan relies on cron. Provides a concrete example: 'classify all of last week's calls in one prompt', guiding agents on applicable scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully bears the burden of behavioral disclosure. It explains optimistic enforcement, caching, in-flight pass, enforceMode behaviors, and scoping rules—all critical for safe invocation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Packs extensive detail into a single paragraph without fluff. Every sentence adds value: scope variants, enforcement modes, error conditions, and example phrasings. Front-loaded with core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 params, scoping, enforcement, caching) and no output schema, the description covers all necessary aspects: side effects, error cases, inter-dependencies, and operational semantics. Nothing obvious is omitted.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 100% schema coverage, the description enriches each parameter with context: inter-parameter relationships (tagKey/tagValue mutual exclusivity, projectId omission for account-wide), example values, and implicit behavior (e.g., 409 for duplicate account gate). This goes well beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it creates a runtime budget gate, specifying the verb and resource. It distinguishes from siblings like delete/update/get budget gates by explaining the creation-specific details, but does not explicitly contrast with them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides extensive context on when to use (creating a budget gate with various scopes) and constraints (e.g., duplicate account gate causes 409, tag params mutually exclusive with projectId). However, lacks explicit directive on when not to use (e.g., to update an existing gate).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses: Pro+ requirement, SSRF defense, URL must be HTTPS, optional secret behavior, eventTypes behavior (omitted/empty = all events), limit of 10 per account, delivery payload format, and signature header. This is highly transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (3 sentences) and front-loaded with the purpose and constraints. Every sentence provides necessary information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is thorough, covering constraints, parameter behaviors, limits, and delivery format. However, with no output schema, it does not explicitly state what the API returns upon success (e.g., webhook ID). This is a minor gap for a creation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by explaining that eventTypes omitted/empty means subscribe to all (though schema also says that) and mentions the account limit of 10, which is not in the schema. It also describes the delivery payload format, which adds context beyond parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool registers an outbound event webhook, using the verb 'Register' and specifying 'Pro+ only, POST /v1/webhooks'. It distinguishes from sibling tools like delete_webhook, update_webhook, list_webhooks, test_webhook by focusing solely on creation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use this tool (creating a webhook) but does not explicitly state when not to use it or list alternative tools. However, the context is sufficient for an agent to infer usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses the cascading delete of eval_scores, permanent loss of historical analysis, scope constraints (Pro+ only, 404 for global/other accounts), and the DELETE method and status code. No annotations present, so description fully covers behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with purpose and method, then provides warnings and alternative. Efficient but slightly lengthy due to necessary warnings; every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given destructive nature, no output schema, and single parameter, the description fully covers side effects, prerequisites, and behavior. Complete for agent decision-making.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% so baseline is 3. The description does not add additional meaning beyond the schema's description of criterionId.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool deletes a custom criterion, distinguishes it from update_eval_criterion for renaming, and specifies the verb 'delete' and resource 'custom criterion'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly warns against casual use, advises only when user confirms not needing history, and directs to update_eval_criterion for renaming. Provides when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully shoulders the burden. It details the baseline window logic, tunable sensitivity with explicit thresholds (1.5x, 2x, 3x), the 4 axes, output structure, errorRate formatting as percent, and an edge case for insufficient data. This is comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single paragraph that efficiently covers purpose, parameters, output, and edge cases without redundancy. It is front-loaded with the core comparative logic. Minor room for restructuring but overall compact and informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description fully specifies the return structure including nested objects, severity levels, ratio, and narrative. It also covers the edge case of insufficient baseline data. For a tool with 2 parameters and a well-defined output, this is complete and self-contained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with both parameters documented, but the description adds meaning: it maps threshold enum values to multiplier meanings (sensitive=1.5x, etc.) and explains that window is the observation period. It also clarifies the 4 axes of detection, which is not in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it compares a current window to a baseline to detect anomalies across 4 specific axes (cost, latency, error_rate, call_volume). This is unique among the sibling tools, which focus on CRUD, alerts, evaluations, etc., making it easily distinguishable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says it lets an AI grasp 'is anything off?' in one prompt, indicating usage for anomaly detection. While it doesn't list when not to use or alternatives, the context is clear and no similar sibling tool exists, making usage intuitive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavior: pagination mechanism (keyset cursor with beforeTriggeredAt + beforeId), null values for unacknowledged events, inclusion of snapshot fields (thresholdValue, windowMinutes, alertType), and that event ids can be passed to acknowledge_alert. This is highly transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized and front-loads the core action. Every sentence adds distinct information (pagination, snapshot, acknowledgment). Slightly longer than necessary but not wasteful. Could potentially be tightened but remains efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, the description covers key return fields (acknowledgedAt/By, snapshot values) and pagination. It explains account-wide vs narrow, the use for acknowledgment, and how to paginate. This is complete for a list tool with no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by explaining the pagination parameters (must be passed together), clarifying that alertId narrows to one alert vs all, and stating the default limit. This enriches the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Return alert trigger events, newest first' and distinguishes account-wide vs narrowed by alertId. It is a specific verb+resource with scope differentiation, clearly distinguishing from sibling tools like list_alerts (which lists alert definitions) and acknowledge_alert.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides example questions that the tool answers ('which alerts fired recently and how often?' and 'when did the cost alert go off?'), giving clear context. It does not explicitly state when not to use it or mention alternatives, but the provided examples effectively guide usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses multiple behavioral traits: it only marks events for retry (actual re-dispatch is manual), uses deterministic idempotency for duplicate detection, requires approval for actual execution, returns specific shapes, emits audit events, and has access restrictions. This is comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with the purpose in the first sentence, followed by selection criteria, behavior, and return shapes. It is somewhat verbose but every sentence adds necessary information. Could be more concise, but it is not overly long for the complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no output schema and six parameters, the description provides complete context: input semantics, behavioral nuances (dry run, duplicates, access control), explicit return shapes for both dryRun true/false, audit details, and idempotency. No gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% parameter description coverage, but the description adds value by explaining how parameters interact (e.g., combining eventIds with fromTimestamp, 7-day cap on range, dryRun behavior, approvalId usage, maxRetries cap). It goes beyond the schema's individual descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the tool marks failed Stripe webhook events for reprocessing, with specific verb 'retry' and resource 'failed Stripe webhook events'. It distinguishes itself from sibling tools like test_webhook by noting it is an internal founder-operations-only recovery tool, and focuses on a narrow use case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly states when to use the tool: to retry failed Stripe webhooks, even providing a common prompt phrase. It mentions access restrictions (403 for general accounts) and internal nature, but does not explicitly list alternative tools or scenarios where it should not be used.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavioral traits: SSRF defense, HMAC-SHA256 signing when secret is provided, rate limit (5/min sliding window, possible exceed), and response field meaning (delivered). No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single dense paragraph covering all key aspects without excess. Could be slightly more structured (e.g., bullet points), but every sentence adds necessary information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description explains the key response field (delivered). All aspects are covered: purpose, constraints, behavior, rate limits, and result interpretation. Complete for a test tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage with descriptions. The description adds value by explaining the purpose of the secret (signature) and the default for alertName, going beyond the schema's basic descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the action ('Send one fabricated alert'), the resource ('to the given URL'), and the purpose ('checking that a webhook URL is reachable before registering it'). It clearly distinguishes from sibling tools like create_webhook or update_webhook.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description specifies the main use case and includes important prerequisites: Pro+ tier, SSRF defense requiring https and rejecting private IPs. It does not explicitly state when not to use, but the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description fully covers behavioral traits: atomic SQL statement, audit logging (bulk_deleted event), cascade delete of related entities via FK constraints, dryRun behavior, and approval flow. Comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with main action, every sentence adds unique information. No repetition or fluff. Efficiently covers purpose, usage, behavior, and side effects.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is remarkably complete: it explains what the tool does, when to use it, how it works internally, side effects, and parameter nuances. Leaves no ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description adds value by explaining dryRun's confirmation purpose and approvalId's server-side verification and atomic consumption, beyond the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states bulk-delete of call ids, scoped to account, with a max of 100. Distinguishes itself via 'garbage calls' context and no sibling tool offers bulk deletion of calls.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives a clear use case ('cleaning up garbage calls from dogfooding/dev tests') and mentions dryRun for safe verification. Does not explicitly state when not to use, but the context is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses cascading delete of related alert_events and irreversibility compared to recoverable alternatives. No annotations provided, so description fully covers behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with main action, then cascade info, then alternatives and precaution. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter delete tool with good schema, description covers relevance: irreversibility, cascading effect, alternatives, and recommended precaution. Complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter (alertId) with complete schema including pattern and description. Description adds 'from list_alerts' context, but schema already has high coverage. Baseline 3 appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Delete an alert' with HTTP method and endpoint, and distinguishes from sibling tools by mentioning alternatives like silence_alert and update_alert.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly describes when to not use (for pausing, prefer silence_alert or update_alert) and recommends checking details with get_alert first. Strong guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses the returned status values and the default-deny security model, and clarifies the relationship with mutation tools. Could mention idempotency or rate limits, but for a simple read tool it is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, entirely relevant, no filler. The key purpose and security note are front-loaded. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple get tool with one parameter and no output schema, the description is fully complete. It covers the return values, the required condition for proceeding, and references the associated request_approval tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and schema already describes approvalId. The description adds helpful context that the ID starts with 'apr_', which aids proper parameter usage. This goes beyond the schema, so above baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves the current state of an approval request, listing all possible status values and referencing the companion request_approval tool, which distinguishes it from siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly warns not to perform the target operation unless status is approved (default-deny), and explains how the approvalId can be used server-side by dangerous mutation tools. This provides clear when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description effectively discloses the tool's behavior: it accepts global and own custom criteria, rejects others with 404, and mentions 'structural defense'. It could further detail response format or auth requirements, but it's sufficient for a simple fetch operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. The purpose is front-loaded, and the contextual guidance follows efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given one parameter, no output schema, and no annotations, the description covers all essential aspects: what it returns, where the ID comes from, and the scope of valid IDs. It is complete and self-contained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a clear description of criterionId. The tool description adds value by explaining the ID's origin (from list_eval_criteria) and the acceptance rules, which goes beyond the schema's autoincrement note.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Fetch' and the resource 'criterion's detail', listing specific fields (name, rubric, scaleMin, scaleMax, createdAt). It distinguishes from sibling tools like list_eval_criteria and update_eval_criterion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly tells where to get the ID ('from list_eval_criteria.criteria[].id') and explains which criteria are accepted (global defaults and own account's customs) and what happens with other accounts' customs (404). This provides clear when-to-use and exclusion guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully covers behavior: it is a read-only operation (implicit by 'Get'), returns specific fields, mentions auto-reset at month boundaries, and states availability to all tiers. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loads the purpose and response fields, and provides usage context efficiently. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description thoroughly explains the response fields and includes important context like the default budget, auto-reset behavior, and usage scenarios. It is complete for an AI agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameters (100% coverage), so the description does not need to document parameters. It adds value by explaining the response structure and usage context beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves the current monthly LLM feature budget, specifies the three axes it covers, and lists the response fields. It distinguishes from sibling tools like 'raise_llm_budget' and 'get_budget_gate' by explicitly stating its read-only nature and purpose for checking budget thresholds.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when the tool is used: by an AI agent deciding if the budget has hit 80% or if it should be raised. It notes availability across tiers. However, it does not explicitly mention when not to use it or compare with alternatives like 'raise_llm_budget' or 'get_budget_gate'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description compensates by listing exact return fields (labels, score, reasoning, classifier_id, source) and noting the endpoint resource template, indicating a read-only GET operation with no side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each earning its place: purpose, usage hint, and return details. Front-loaded and free of redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter tool with no output schema, description fully covers purpose, input sourcing, and output structure. No gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema already describes assessmentId as 'AUTOINCREMENT integer'. Description adds value by specifying where to obtain the id (from list_safety_assessments), providing usage context beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Fetch one assessment's detail by id', specifies verb and resource, and distinguishes from list counterpart by referencing list_safety_assessments. It also outlines return fields, providing complete purpose clarity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit instruction to use assessments[].id from list_safety_assessments, leaving no ambiguity about input source. Does not explicitly state when not to use, but given the singular nature of the tool, guidance is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that Free plan can read all criteria but creating custom ones is Pro+, and existing rows remain visible after downgrade. No annotations exist, so description fully carries the burden. For a read-only listing tool with no parameters, this is highly transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, no fluff. Front-loaded with main purpose, then details and usage guidance. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and no output schema, the description is complete: explains output fields, global vs custom, plan limitations, and usage context. Nothing missing for this simple listing tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% but there are no parameters. The description adds meaning by detailing the output fields (id, name, rubric, scaleMin, scaleMax) and the distinction between global and custom criteria. This compensates for the lack of parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it returns the list of LLM-as-judge evaluation criteria, including 5 global defaults and custom criteria, with specific fields (id, name, rubric, scaleMin, scaleMax). This distinguishes it from sibling tools like create_eval_criterion, get_eval_criterion, and others.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit usage guidance: 'Use before running an eval to see which axes are available.' Also notes plan limitations (Free vs Pro+), helping set expectations. No explicit exclusion of alternative tools, but context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes zero side effects, data source (call patterns over lookbackDays), and output structure. No annotations, so description carries burden; it covers key behavioral traits adequately.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Relatively concise for the information provided. Front-loaded with main action and distinction from create_alert. Each sentence adds value, though could be slightly trimmed without losing meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, description fully explains the return object including fields like baseline, proposals, skipped. Parameter is fully described. No missing context for an agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Single parameter with 100% schema coverage. Description adds range, default, and context: 'analyze the call patterns over the past lookbackDays (7-30, default 14).' Also explains how lookbackDays appears in output.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it analyzes call patterns and proposes alert rules for cost/latency/error_rate/anomaly. Distinguishes from sibling tools like create_alert and list_alerts by emphasizing the propose-only nature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly separates proposal from creation: 'Applying them is a separate step via create_alert after customer confirmation.' Also notes that only non-overlapping rules are proposed, referencing list_alerts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description fully discloses behavioral traits: irreversibility (NULLing plaintext), need for human approval via request_approval, idempotency semantics with hash, dryRun behavior, and audit event emission. This is comprehensive given no annotations are provided.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose and then provides supporting detail. It is slightly verbose with technical details (e.g., hashing algorithm) but remains efficient overall without unnecessary repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (3 parameters, no output schema, destructive action) and 45 sibling tools, the description covers all essential aspects: purpose, parameters, behavioral traits, prerequisites, error conditions, return values, and audit. It leaves no major gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 100% schema description coverage, the description adds significant value: it explains the safe default for dryRun, the pattern and creation of approvalId, the default and range for olderThanDays, and the interplay between parameters. This goes beyond the schema's basic descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool bulk-purges plaintext records older than a specified number of days, with the specific verb 'purge' and resource 'plaintext records'. It distinguishes from sibling tools by focusing on plaintext retention policies and mentioning account-level operation, which is distinct from other deletion tools like bulk_delete_calls or delete_alert.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit when-to-use guidance, including the example of auto-purge older than 30 days, and when-not-to-use for Free plan users (403). It also recommends using dryRun=true for preview. However, it does not explicitly contrast with other deletion tools, though the unique approval requirement implicitly differentiates.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description fully discloses behavioral traits: feeds inputText to targetModel, uses gpt-4o-mini for scoring against criteria, records eval_scores, and explains idempotencyKey for dedup. It also notes cost implications and environment dependencies.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (about 100 words) and well-structured, with front-loaded main action. Every sentence adds unique information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, the description covers essential aspects: what the tool does, how it works, cost model, idempotency, error conditions, and comparison with sibling tools. It is complete for a mutation tool with 4 parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds value by explaining default for judgeModel, restrictions for targetModel (OpenAI models only), and the purpose of idempotencyKey (retry dedup and double-billing prevention).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool runs a golden dataset against a target model to produce a regression verdict. It specifies the HTTP method and endpoint, distinguishes from siblings like compare_eval_runs, and mentions the Pro+ plan restriction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit context: results can be compared with compare_eval_runs, run records are excluded from production cost/analytics/alert aggregation, and it warns about cost and 503 errors without API key. It doesn't explicitly state when not to use, but offers strong guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behaviors: error codes (403, 409), idempotency via tier2_idempotency table, delegation to Stripe, atomic consumption of approvalId, and dryRun semantics. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with purpose and restrictions, then details parameters and behaviors. It is well-structured but slightly verbose; some sentences could be merged. However, every part adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description covers purpose, error conditions, parameter details, and idempotency. It lacks explicit return format but mentions preview output for dryRun. Adequate for the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
While schema coverage is 100%, the description adds significant context beyond schema: dryRun effects, idempotencyKey requirement, approvalId consumption pattern, targetAccountId restriction. It explains parameter interactions and constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool applies a promotion code to a Stripe subscription, specifying the resource (promotion code), action (apply), and constraints (registered in Stripe, user-facing). It distinguishes from siblings as there is no other promo-related tool in the list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly limits usage to founder-operations (403 for others), explains when not to use (active discount or canceled status), and provides guidance on dryRun vs actual application. No comparison to siblings needed as no alternative exists.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, but description fully covers behavior: nothing is inserted, decrypts samples, sends excerpts to OpenAI, results are advisory, includes return structure and audit event.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is detailed but front-loaded with key information. Each sentence adds value; however, it could be slightly more concise without losing clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description thoroughly explains return structure, including partialFailures and budgetSpentUsd. Covers edge cases like decrypt failures. Complete for a 3-param tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description adds value by explaining the purpose of each parameter and behavioral aspects (e.g., sampleCallIds grounds proposals).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool proposes eval criterion candidates using an LLM judge from a useCaseHint and optional sampleCallIds. It distinguishes from sibling create_eval_criterion by noting adoption is a separate step.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use ('propose criteria to measure our prompt quality'), mentions prerequisites (Pro+ only), provides alternative for privacy (useCaseHint only), and recommends user review before adoption.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses mutation behavior, session authentication with CSRF protection (Origin/Referer), and plan-based restrictions. With no annotations provided, the description fully covers behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise single sentence that first states the core purpose, then efficiently adds plan details and auth requirements. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Fully covers all relevant context: action, parameters, plan restrictions, and authentication behavior. No output schema needed for this mutation tool; description is complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with good descriptions. The description adds clarity by explaining 'name = display name' and 'slug = a short URL-safe identifier', but mostly echoes schema info. Slight added value over schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action 'Create a new project' and specifies the HTTP method. Distinguishes from sibling create tools by focusing on project-specific details like plan limitations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit usage context including plan caps (Pro 5, Team unlimited, Free cannot create) and authentication requirements. This helps the agent decide when to use this tool and when not to.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses the physical deletion, no restore, and the effect on eval_runs (setting prompt_registry_id to NULL, losing trace). Provides comprehensive behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with vital information front-loaded. Every sentence adds value; no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, but description explains the 204 response and side effects on eval_runs. Adequately covers all aspects given the tool's complexity and context from sibling tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with parameter description, but the description adds contextual semantics like 'Scoped to your account' and references list_prompts, adding value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Delete an existing prompt' and specifies the REST endpoint and response code. Distinguishes from siblings by emphasizing physical deletion and comparing to update_prompt for logical sunset.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly warns that deletion is physical and irreversible, and advises using update_prompt with labels for safer sunsetting. Also notes scoping to user's account.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It details the dryRun behavior (preview vs mutation), idempotency key caching, cumulative cap, Stripe mutation side effects, and approvalId atomic consumption. Comprehensive disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Dense single paragraph with all essential information front-loaded. Every sentence adds value. Slightly long but appropriate for the complexity; could be broken into bullet points for readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, yet the description explains return values for dryRun (preview of dates and cumulative total) and the actual mutation effects. Covers error conditions (403, 409) and status checks. Complete for a sensitive financial tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds critical context beyond schema: dryRun must be explicit, idempotencyKey required when dryRun=false, cumulative cap explanation, and atomic approvalId consumption. Enriches all parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'extend' and the resource 'Stripe subscription trial' with a specific range (1-30 days). Includes the HTTP endpoint and distinguishes itself from sibling tools by noting it's an internal support tool for founder-operations only.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use (founder-operations, status='trialing'), when not to use (general accounts get 403), and limitations (cumulative cap of 60 days). Provides clear context for usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Discloses creation of request, human approval via email/dashboard, timeout expiry counted as denied, server-side consumption with specific mutation tools, action matching, and one-time use. Fully transparent about side effects and constraints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with purpose, usage example, key notes, and consumption details. Every sentence is informative, though slightly lengthy. Front-loaded with core action and context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex tool with 4 params, nested objects, and no output schema, the description is remarkably complete. Covers workflow, expiration, consumption, action matching, and dependencies. Leaves no critical gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, but description adds significant value: explains action should match target tool name for automatic consumption, summary appears verbatim in email, timeoutSeconds default 3600, and metadata usage. This extra context goes beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Create an approval request in the human approval gate' with context (Phase 3, Pro+ only) and examples. Distinguishes from siblings by mentioning polling with get_approval and listing dangerous operations that require it.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Call it before dangerous operations' and lists examples. Warns that no MCP tool exists for self-approval and instructs to poll with get_approval. Provides clear when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavior: PATCH method, partial updates (all fields optional), enabled flag behavior (pauses vs. silence), and conditions override behavior. Covers mutation safety and lifecycle.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise single paragraph (~70 words) front-loading the verb+resource, then key constraints, then examples. No wasted words, efficient structure.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers all critical aspects: what updates, what is immutable, optionality, field behaviors, and example usage. Given 11 parameters and nested objects without output schema, this is complete and actionable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with detailed descriptions. The description adds value by highlighting immutability of alertType, optionality of all fields, and the conditions override. While schema is already strong, the description provides context beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it updates an existing alert's settings, identifies the immutable alertType, and lists mutable fields. It distinguishes from sibling tools like create_alert and delete_alert by specifying the update lifecycle.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance: alertType is immutable, instructing to create a new alert and delete the old one. Offers example phrasing for typical use cases like lowering a threshold or adding a notification channel.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses partial update behavior (only passed fields updated), immutability constraints, and the nuance about description field (rejecting empty string to prevent accidental wipe). No annotations exist, so description fully addresses behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two well-structured sentences, front-loaded with key information, no wasted words. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given complexity (5 params, nested objects) and absence of output schema, the description covers operation, restrictions, usage scenarios, and important edge cases (description clearing). It is complete for an AI agent to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with good parameter descriptions. The tool description adds context like 'variables (null clears them all)' and 'promptId is required', which supplements the schema meaningfully.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it's for partially updating an existing prompt, lists the updatable fields (template, variables, labels, description), and distinguishes from rename_prompt. It specifies the HTTP method and Pro+ requirement.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly warns that name and version are immutable and must be changed via rename_prompt. Provides usage examples like label moves and small patch edits, guiding when to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavior: per-event action, idempotent (does not overwrite ack info), return 200 with flag. No hidden traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with main action, then contrast, then details. Every sentence earns its place with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, description explains return behavior (200, flag) and idempotence. Sufficient for a simple acknowledge operation. Sibling tools cover related alert actions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema already describes eventId with origin note; description adds 'Pass the id obtained from list_alert_events as eventId', reinforcing correct usage. High schema coverage (100%) plus extra context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it marks an alert event as acknowledged, and explicitly distinguishes from sibling silence_alert by explaining per-event vs rule-level behavior.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance on when to use this tool (per-event ack) versus silence_alert (rule-level mute), and explains idempotent behavior with the alreadyAcknowledged flag.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![mcp-server MCP server](https://glama.ai/mcp/servers/argosvix/mcp-server/badges/card.svg)](https://glama.ai/mcp/servers/argosvix/mcp-server)

Score Badge

Copy to your README.md:

[![mcp-server MCP server](https://glama.ai/mcp/servers/argosvix/mcp-server/badges/score.svg)](https://glama.ai/mcp/servers/argosvix/mcp-server)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/argosvix/mcp-server'

If you have feedback or need assistance with the MCP directory API, please join our Discord server