OpenSIPS MCP Server

Official

by OpenSIPS

Overview Schema Related Servers Score Discussions

Python

Hybrid

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v1.0.0
Disambiguation4/5
Most tools are clearly scoped by module prefix (e.g., b2b_, cfg_, drouting_) and have distinct purposes. However, the sheer number (236) creates some overlap, e.g., multiple 'list' and 'reload' tools for different subsystems that could confuse an agent.
Naming Consistency5/5
Tool names follow a highly consistent noun_verb pattern with module prefixes (e.g., address_add, subscriber_create). A few minor inconsistencies (acc_query_cdrs vs accounting_query_cdrs) are present but do not detract from overall uniformity.
Tool Count2/5
236 tools is far beyond the recommended range for most servers. While OpenSIPS is a complex system, the set is very large and could be streamlined; many functions might be redundant or better consolidated.
Completeness5/5
The tool surface covers nearly every aspect of OpenSIPS administration: config generation, migration, linting, monitoring, testing, data management, and module development. It is hard to identify significant gaps for the stated purpose.
Average 3.7/5 across 200 of 236 tools scored. Lowest: 2/5.
See the Tool Scores section below for per-tool breakdowns.
- No community issues in the last 6 months
- 0 commits in the last 12 weeks
- No stable releases found
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI is passing
This repository is licensed under Apache 2.0.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
Add a glama.json file to provide metadata about your server.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description must disclose behavioral traits. It fails to mention whether updates are partial or full, whether the rule must exist, if changes take effect immediately, or any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (one sentence) but lacks structure. It earns its place but does not address the tool's complexity, making it insufficiently informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 14 parameters and no output schema description, the description is wholly incomplete. The agent has no guidance on parameter interplay, return values, or usage patterns.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds no meaning to the 14 parameters. The agent must rely solely on schema titles, which lack context for usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Update an existing fraud detection rule' clearly states the verb (update) and the resource (fraud detection rule), distinguishing it from sibling tools like fraud_add_rule and fraud_delete_rule.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines1/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., fraud_add_rule vs fraud_update_rule), nor are prerequisites or when-not-to-use scenarios mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description fully bears the burden of behavioral disclosure. It only states an action ('add') without detailing side effects, success conditions, error states, or idempotency. This is insufficient given the lack of annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
While a single sentence is concise, it is under-specified. The description should include more details to be useful, violating the principle that every sentence must earn its place. Brevity here sacrifices clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 13 parameters, no parameter documentation, and no hints about the output schema, the description is grossly inadequate. The tool likely requires significant context (e.g., profileid meaning, threshold semantics) that the description omits.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning the description adds no information about parameters. It does not explain the meaning of fields like profileid, prefix, cpm_warning, etc., leaving the agent to guess their purpose.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Add a new fraud detection rule' clearly states the action and resource, distinguishing from sibling fraud tools like delete, list, update, and reload. However, it lacks any additional context that would differentiate it from similar 'add' tools for other entities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as fraud_update_rule or fraud_reload. The description does not mention prerequisites or scenarios where adding a rule is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It only says 'Add to database' without disclosing behavior on duplicates, idempotency, side effects, or if reload is required. Critically lacking.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence is very concise but overly terse, omitting crucial information. It does not earn its place given the gaps it leaves.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one required param and output schema, the description should at least mention return value or common use cases. It fails to provide a minimal complete picture.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but description adds no meaning beyond the schema itself. 'domain' and 'attrs' are not explained at all, failing to help the agent understand their purpose or format.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'Add' and resource 'SIP domain', and distinguishes from siblings like domain_delete and domain_list. It specifies the action and target resource accurately.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No indication of when to use this tool versus alternatives (e.g., checking if domain exists with domain_list, or needing domain_reload after adding). Missing guidance on prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should disclose behavioral traits. It only indicates a read operation ('List') but omits pagination behavior, ordering, or any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise, but it sacrifices necessary detail for brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description does not clarify what the returned data contains, leaving the agent without sufficient context for a list operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain the limit and offset parameters, nor their implications, failing to add value over the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'List dynamic routing carriers,' which clearly identifies the verb and resource, but does not differentiate from sibling tools such as drouting_list_gateways or drouting_list_rules.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives; lacks context about prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must disclose behavioral traits. It only says 'Update' without explaining side effects (e.g., password rehashing), idempotency, or that only provided fields are updated (partial update). Insufficient for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence front-loading purpose. Very concise, but lacks necessary detail. Not excessively wordy, but could include more information without being verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 3 params, no nested objects, and an output schema (not shown), the description is too minimal. It does not clarify optional fields (partial update), allowed values, or how it differs from subscriber_create (e.g., required id). Incomplete for reliable agent decision-making.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. Description mentions 'password or attributes' but the only additional param is email_address; 'attributes' is ambiguous. Does not explain the purpose of id (required integer) or constraints on email/password format. Does not compensate for missing schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose3/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Update a subscriber's password or attributes.' It specifies the verb 'Update' and resource 'subscriber,' but 'attributes' is vague; the schema only shows password and email_address. It distinguishes from subscriber_create and subscriber_delete but lacks precision.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives like subscriber_bulk_import for bulk updates, or subscriber_create for creating. No prerequisites or conditions mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description only mentions 'update' without disclosing behavioral traits like required permissions, destructive nature, or the meaning of the 'reload' parameter.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
While short, the description is under-specified for the complexity of the tool; it fails to provide necessary details, making it insufficiently helpful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 12 parameters, no schema descriptions, no annotations, and only a one-sentence description, the tool is severely incomplete for an agent to use effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds no meaning to the 12 parameters beyond their names; it only mentions updating by id.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Update a dialplan rule by id' with a specific verb and resource, distinguishing it from sibling tools like add, delete, list, reload, and translate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives; lacks context such as prerequisites or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description only states the action without disclosing behavioral details such as whether deactivation is reversible, if it drains existing calls, or if it requires specific permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (one sentence), which is good for conciseness, but it lacks necessary detail for a tool with 4 parameters. It is under-specified rather than efficiently comprehensive.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (4 required params, no annotations, no output schema details in description), the description is incomplete. It fails to provide parameter meanings, return value, or behavioral context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning the description does not explain any of the 4 required parameters (carrier, domain, prefix, host). The description adds no meaning beyond the schema titles.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Deactivate a carrier route host' and further clarifies with parenthetical 'drain, take out of service'. This provides a specific verb and resource, and distinguishes from sibling tools like carrierroute_activate_host.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives such as carrierroute_replace_host or carrierroute_activate_host. The description does not mention when not to use it or any prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, and the description fails to disclose key behavior: what happens on invalid callid, whether existing variables are overwritten, side effects, or error handling. Minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short and uses a bullet list for parameters, but it could be more concise by integrating the parameter list into a single flow. Still, it avoids unnecessary text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (4 params, all strings) and presence of an output schema, the description lacks information on return value, error states, and success conditions. Incomplete for an agent to use reliably.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description merely restates parameter names without adding any meaning (e.g., format, constraints, dependencies). The parameter list is copied from schema with no extra value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Push a variable into an active dialog' uses a specific verb ('Push') and resource ('variable into an active dialog'), clearly distinguishing it from sibling dialog tools like dlg_end, dlg_list, and dlg_get_profiles.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines1/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance provided on when to use this tool vs alternatives, prerequisites (e.g., dialog must be active), or scenarios to avoid. Completely lacking usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so description should disclose behavioral traits. It states 'update' implying mutation, but does not mention side effects, permissions, or the impact of parameters like 'reload'. Lacks behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise (one sentence), but underinformative. Front-loaded with purpose, but lacks necessary details for parameter understanding and usage. Conciseness comes at cost of completeness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (11 parameters, no schema descriptions, no annotations), the description is severely incomplete. An agent cannot correctly use this tool without guessing parameter meanings or behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description must add meaning. It only mentions 'gwid', leaving the other 10 parameters (address, type, strip, etc.) unexplained. No value added beyond the schema's basic types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action ('Update'), resource ('dynamic routing gateway'), and key identifier ('by gwid'), distinguishing it from sibling tools like drouting_add_gateway and drouting_delete_gateway.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, nor any when-not or context regarding prerequisites or conditions. The description is purely operational.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The only behavioral trait mentioned is the optional trigger of MI dr_reload. No annotations exist, so the description carries full burden. Missing information on side effects, permissions, idempotency, or reversibility.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no unnecessary words. However, it is too concise and lacks critical details, so it cannot be a 5.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 13 parameters, no annotations, and no output schema details, the description is insufficient for an agent to correctly invoke the tool. It omits return values, parameter semantics, and behavioral aspects beyond the basic action.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the tool description adds no explanation of parameters beyond the schema titles. The description only references ruleid, ignoring other important params like groupid, prefix, gwlist, etc.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update a dynamic routing rule by ruleid' which specifies the action and resource. However, it does not differentiate from siblings like drouting_add_rule or drouting_update_carrier, which also operate on dynamic routing entities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. No prerequisites, contexts, or exclusions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It states the action but does not disclose behavioral traits like persistence across restarts or required permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no fluff, but its brevity leads to omission of crucial information, making it less effective than a longer but more informative description.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is too minimal for a tool with two parameters and no schema descriptions. It lacks context on prerequisites, parameter details, and output, leaving the agent underinformed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description adds no parameter-level meaning. The agent cannot infer the format of 'ip' or valid 'jail' values from the description or schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('ban') and resource ('IP address') within a clear context ('fail2ban jail'). It effectively distinguishes from the sibling tool 'fail2ban_unban_ip'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as when a ban might already be in effect or prerequisites like jail existence.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should disclose behavioral traits. It only states the action without mentioning idempotency, authentication, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: a one-line summary followed by a parameter list. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description does not explain return values or behavior on success/failure. For a state-changing tool, this is insufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, so the description must compensate. It provides brief explanations for each parameter (group, destination, enabled), adding basic meaning beyond types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool enables or disables a load balancer destination, specifying the verb and resource. It distinguishes from sibling tools like lb_reload or lb_list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines1/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. It does not mention prerequisites, scenarios, or when to avoid using it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations; description does not disclose pagination behavior (limit/offset), default ordering, or any side effects. Agent cannot infer that results are paginated or that domain filter is optional.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence is concise but lacks depth. Front-loaded with purpose, but omits essential details about parameters and behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Complexity moderate (3 params, output schema). Description fails to cover pagination, defaults, or use cases. Incomplete for a listing tool with multiple parameters and siblings.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage 0%—no parameter descriptions in schema. Description only mentions 'domain' filter, ignoring limit and offset parameters. Leaves 2 out of 3 parameters unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'List subscribers' with optional domain filter. Distinct from sibling tools like subscriber_get (single) and subscriber_create. Missing explicit differentiation from subscriber_bulk_export.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives. Does not mention pagination limits or when to use domain filter. Siblings exist but no differentiation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits, but it only says 'Add' without indicating permissions, reversibility, or impact. The 14 parameters are not explained.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise (one sentence) but under-specified. It is not structured and provides minimal information beyond the tool name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (14 parameters, no annotations, output schema present), the description fails to provide essential context for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and the description adds no meaning to any of the 14 parameters. The agent gets no help on how to fill in parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Add') and the resource ('new TLS domain configuration'), distinguishing it from sibling tools like tls_update or tls_delete.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like tls_update or tls_list. No prerequisites or context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so description carries full burden. It does not disclose behavioral details such as idempotency, side effects, or state changes. Merely states 'enable', which is insufficient for an agent to understand consequences.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise (6 words), but at the expense of clarity. It earns its place but could benefit from additional context without being verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and an output schema, the description is too minimal. It lacks information about return values, side effects, or conditions, making it incomplete for an agent to invoke confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% (no parameter descriptions). The description mentions 'by AOR' but does not explain what an AOR is or any format constraints, forcing the agent to rely solely on the parameter name.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description uses a specific verb ('Enable') and resource ('UAC registration by AOR'), clearly distinguishing from sibling tools like uac_reg_disable. However, it does not elaborate on what 'enable' entails.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like uac_reg_disable or uac_reg_force_register. No context on prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without any annotations, the description must disclose behavioral traits. It only says 'Delete', which implies destructiveness, but does not mention whether deletion is permanent, whether confirmation is needed, or any side effects like cascading removal of associated data. This is insufficient for a destructive operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is extremely concise but under-informative. It lacks structure such as bullet points or front-loading of critical information (e.g., that the operation is irreversible). Conciseness is achieved at the expense of completeness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is a destructive single-parameter operation but lacks crucial context: return value (despite having an output schema), error conditions (e.g., domain not found), or confirmation steps. The description is not complete enough given the tool's simplicity and the presence of an output schema that is not explained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, meaning the description does not elaborate on the 'domain' parameter. The description merely restates what the schema already shows. It adds no additional meaning, such as format expectations or examples, leaving the agent to guess the exact domain identifier required.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (Delete) and resource (SIP domain), and it effectively distinguishes from sibling tools like 'domain_add' and 'domain_list', which are for creating and listing domains respectively.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool, no prerequisites (e.g., domain must exist), and no conditions under which deletion should be avoided. Absent any usage context, an AI agent must infer from name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The only behavioral trait disclosed is that it 'optionally triggers MI dr_reload,' but the description does not clarify that the default is true (from schema) or explain side effects. No annotations provided, so burden is on description, which is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short and front-loaded, but it omits critical information, so it does not fully earn its place. It could be more informative without sacrificing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 8 parameters, no schema descriptions, no annotations, and a family of drouting tools, the description is incomplete. It does not mention return values, success/failure indicators, or behavior on duplicate carrierid.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description adds no value for parameters. It does not explain required parameters like 'gwlist' or optional ones like 'flags' or 'sort_alg'. Only 'reload' is hinted but not clarified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Add a new dynamic routing carrier' (verb+resource), which distinguishes it from sibling tools like drouting_add_gateway and drouting_add_rule. No ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives or prerequisites. The description does not mention when not to use it or any conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description only states 'delete' without explaining consequences, such as whether the deletion is reversible, if it triggers a reload (though a reload parameter exists), or if it requires specific permissions. With no annotations, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at one sentence, but it effectively communicates the core purpose. No unnecessary words, though it could be expanded slightly without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and two parameters (one with a default that suggests behavioral impact), the description lacks critical context such as what happens after deletion, whether the reload parameter is necessary, and any dependencies or prerequisites.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%; the description does not explain the purpose of the 'id' or 'reload' parameters. The schema itself provides basic info (type, default), but the description adds no value beyond that.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (delete) and the resource (dynamic routing carrier by id). It distinguishes from siblings like drouting_add_carrier or drouting_list_carriers, but does not elaborate on what a dynamic routing carrier is or the context of deletion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance provided on when to use this tool versus alternatives (e.g., drouting_update_carrier to modify instead of delete). No mention of prerequisites or side effects.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behaviors. It only states 'Delete', implying destructiveness, but fails to explain side effects, reload behavior, or error conditions. Crucial context is missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise but under-specified. It does not waste words, but the brevity sacrifices necessary detail, making it only adequate for a minimal viable description.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description lacks crucial context for a deletion tool: no mention of required permissions, impact on associated data, return values (despite an output schema), or when reload is needed. The agent would need additional information to use it safely.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, yet the description adds no explanation for either parameter. 'gwid' and 'reload' are not described beyond their schema types and default, leaving the agent without guidance on values or behavior.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete'), the resource ('dynamic routing gateway'), and the identifier ('by gwid'). It distinguishes from sibling tools like drouting_delete_carrier and drouting_delete_rule by specifying the resource type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, no prerequisites or consequences mentioned. The description does not help the agent decide between this and similar deletion tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations; description does not disclose side effects (e.g., reload triggers), error conditions, or security requirements. Minimal information for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One short sentence is concise but under-specified. It could include more structure without sacrificing brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description fails to cover the tool's behavior, parameter semantics, or usage context, making it incomplete for effective selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%; the description adds no explanation for any of the 9 parameters, leaving the agent to infer from schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'update' and the resource 'dynamic routing carrier', and specifies identification by id. It distinguishes from sibling tools like drouting_add_carrier and drouting_delete_carrier.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives (e.g., drouting_add_carrier), no prerequisites, and no context on required fields beyond the schema.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only states the action without disclosing potential side effects, permission requirements, or what happens if the IP is not banned. The behavioral impact is implicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (one sentence), which is concise but sacrifices necessary detail. It is front-loaded but lacks depth, making it only moderately effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the low schema coverage and no annotations, the description is incomplete. It fails to provide context on IP format, jail usage, or expected behavior. The presence of an output schema is not leveraged.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, and the description does not explain the parameters 'ip' or 'jail' beyond their names. The default value for 'jail' is not mentioned. The description adds no meaning to the parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (unban) and the resource (IP address from a fail2ban jail). It distinguishes from sibling tools like 'fail2ban_ban_ip' and 'fail2ban_list_banned' by specifying the unban operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, such as 'fail2ban_ban_ip' for banning or 'fail2ban_list_banned' to check banned IPs. No prerequisites or context for invocation are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It only says 'Resize the capacity', leaving out important behavioral traits such as whether the operation is synchronous, requires downtime, or has any other side effects. The agent lacks context for safe invocation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, consisting of a one-sentence summary followed by a parameter list. It avoids unnecessary words and is structured clearly. However, it could be more compact by integrating parameter descriptions into prose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema and 4 required parameters for a state-modifying operation, the description lacks completeness. It does not explain return values, error conditions, or what happens during resizing. More context is needed for an agent to use it safely.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It lists each parameter with a single-line description (e.g., 'The destination group ID'), but these add minimal meaning beyond the schema property titles. The meaning of 'destination' and 'resource' in the context of resizing is unclear.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('resize') and the target resource ('load balancer capacity'). It distinguishes the tool from siblings like 'lb_list' and 'lb_reload' by specifying the operation on capacity. However, it could be more precise about what 'capacity' means.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool or alternatives. There is no mention of prerequisites, potential side effects, or when not to use it. The description solely states the operation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behavior. It only states 'Delete' (implying mutation) but fails to describe effects like idempotency, error handling (e.g., attempting to delete a non-existent AVP), or whether the action is reversible. No output schema details are mentioned.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no unnecessary words. It is front-loaded with the key information. However, it could be slightly expanded to include parameter details without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite low complexity (one required parameter), the description omits important context: parameter semantics, return value (output schema exists but not explained), and behavioral guarantees. This leaves the agent with insufficient information to invoke the tool confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The sole parameter 'name' lacks a schema description (0% coverage). The description adds no meaning beyond the parameter name – it does not explain what 'name' refers to (e.g., AVP key format, uniqueness constraints).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Delete a global AVP.' uses a specific verb ('Delete') and resource ('AVP' – attribute-value pair), and clarifies scope ('global'). It clearly differentiates from siblings avp_get and avp_set by focusing on deletion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No information is provided about when to use this tool versus alternatives like avp_get or avp_set. There is no mention of prerequisites, when-not to use it, or which sibling to prefer.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description only states basic action without disclosing side effects (e.g., overwrite behavior, permissions, or errors).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with key information. No wasted words, but could be slightly more concise as one sentence.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Missing context on return value, error handling, and scope implications. Inadequate for a tool that sets a value in a global store.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. Description adds no meaning beyond parameter names; no explanation of name or value formats.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states verb 'Set', resource 'AVP value', and scope 'global'. Distinguishes from siblings like avp_delete and avp_get.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. No mention of prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are absent, so the description must disclose behavior. It mentions the optional MI dr_reload trigger (a side effect) but fails to cover prerequisites, authorization, or whether the operation is idempotent. For a mutation tool, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short and front-loaded, wasting no words. However, it is arguably too concise, missing critical details. Still, it earns a 4 given the lack of verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (12 parameters, 3 required, mutation, output schema present), the description is incomplete. It does not explain return values, conflict handling, or the effect of the 'reload' parameter, relying solely on the schema which lacks descriptions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds no explanation for any of the 12 parameters. Parameter names like 'sort_alg', 'timerec', and 'attrs' are not self-explanatory, leaving the agent without guidance on their meaning or format.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Add a new dynamic routing rule' with a specific verb and resource. It distinguishes from sibling tools like drouting_add_carrier or drouting_add_gateway, which operate on different entities within dynamic routing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternative tools such as drouting_update_rule or drouting_add_carrier. The description only mentions an optional reload trigger, not when this tool is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits, but it does not mention whether the operation is read-only, destructive, or requires authentication. It also omits details about pagination, limits, or output format, leaving significant gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description front-loads the purpose but dedicates a long parameter list. It is adequately structured but could be more concise by omitting obvious parameter explanations or integrating them into a shorter form.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters and an existing output schema, the description does not explain the output structure or common usage patterns. It provides only basic filter descriptions, leaving the agent uninformed about expected results or limitations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It provides a one-line explanation for each of the 5 parameters (e.g., 'caller: Filter by calling party (From user)'). While this adds some meaning, it largely restates what parameter names imply, and lacks format constraints or examples.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool searches SIP calls with filters for caller, callee, call_id, and time range. However, it does not differentiate from similar tools like homer_search_calls_v7 or homer_correlate_callid, which is a missed opportunity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It lacks any conditional or exclusionary language, leaving the agent to infer usage from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It only says 'Delete', which implies mutability, but does not mention effects on the system, whether reload is needed, permissions, or undo capabilities. The reload parameter's effect is not explained.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, consisting of a single sentence. However, this conciseness sacrifices important details, making it less useful. It is not well-structured for maximum information retrieval.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 params, no enums), the description should still cover output schema or confirm it. It fails to provide completeness: no info on return values, error states, or how the tool fits with the sibling dialplan tools. The presence of an output schema is not leveraged.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not elaborate on any parameters. It mentions 'by id' but does not explain the 'reload' parameter (default true). The agent cannot infer the meaning or usage of parameters beyond their types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Delete', the resource 'dialplan rule', and the method 'by id'. This distinguishes it from sibling tools like add, list, update, reload, and translate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, no prerequisites or side effects mentioned. The description does not help the agent decide if deletion is appropriate or what to expect.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and description does not disclose behavioral traits like read-only, authorization needs, or output format. Missing details beyond the basic operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, efficient and front-loaded. However, could include brief parameter explanations without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool is simple with optional filter and pagination; output schema exists. Missing parameter descriptions and usage context, but adequate for a basic list operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but description only references 'set ID' without explaining the purpose of 'limit' and 'offset' parameters. Does not compensate for missing schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description uses specific verb 'list' and resource 'dispatcher destinations from the database', clearly distinguishing this read operation from sibling dispatcher mutation tools like dispatcher_add, dispatcher_remove, and dispatcher_update.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus other list tools or dispatcher tools. Does not mention alternatives or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. It only says 'tear down' without explaining side effects (e.g., SIP messages sent, dialog state changes, or whether the dialog must be active). This lacks sufficient transparency for a destructive operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise: a clear one-liner followed by a parameter list. However, the parameter list essentially duplicates the schema, which could be omitted if not adding value. Otherwise, it is well-structured and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that an output schema exists (though not shown), the description need not explain return values. However, for a SIP dialog termination tool, it lacks context about prerequisites (e.g., dialog must be active) and does not indicate behavior in error cases. Sibling tools suggest a rich ecosystem, but no cross-references are provided.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description merely lists parameter names (callid, from_tag, to_tag) without adding any meaning beyond the schema. It does not explain what these tags represent or how to obtain them, failing to compensate for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'End (tear down) an active dialog,' specifying the action (end/tear down) and the resource (active dialog). This distinguishes it from sibling tools like dlg_list or dlg_get_profiles, which have different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as using b2b_terminate for B2B sessions or other termination methods. It does not mention prerequisites or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden for behavioral disclosure. It only states that the tool restarts a service, but does not explain whether the restart is graceful, what happens to existing connections, or any other behavioral traits. This is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise, using a single sentence and a clear parameter listing. No unnecessary words, but the docstring-like formatting could be more natural for an agent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (one parameter with default, output schema exists), the description is minimally adequate. However, it lacks details like potential errors, return value description, or behavior when the service does not exist, which affects completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description must compensate. It does provide a brief explanation of the 'service' parameter and its default, but it lacks details like valid values or whether the service name is a Docker Compose service name. It adds some value but not enough for a high score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Restart' and the resource 'Docker Compose service', which aligns with the tool name. However, it does not differentiate from sibling tools like docker_deploy_scenario or docker_status, which have similar scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidance is provided. There is no mention of when to use this tool over alternatives, prerequisites (e.g., service must be running), or potential side effects. The description simply repeats the action without context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description mentions the optional MI dr_reload trigger, but with no annotations, it fails to disclose other behavioral traits such as persistence, side effects on existing configurations, or required permissions. This is insufficient for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the primary purpose. Every word is necessary, with no redundancy or extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description provides no insight into return format, error conditions, preconditions, or how the reload trigger works. For a tool with 10 parameters, it leaves significant gaps in understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 10 parameters with 0% description coverage, and the description adds no explanation for any parameter. The agent receives no semantic context beyond the schema defaults and field names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Add' and the resource 'new dynamic routing gateway', distinguishing it from sibling tools like drouting_add_carrier and drouting_add_rule. The optional reload trigger is also mentioned, adding specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. The description does not mention prerequisites, when-not-to-use, or distinguish from related tools like drouting_delete_gateway or drouting_update_gateway.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries the full burden. It only states 'delete' with no details on side effects, error handling, return value, or whether the operation is irreversible. The 'reload' parameter is not explained.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. However, it sacrifices important detail, making it slightly too brief for effective use.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and an output schema (unknown content), the description omits behavioral and return-value context. A delete operation warrants details on success/failure indicators and the meaning of the 'reload' parameter.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and the description adds no meaning beyond the schema. 'ruleid' is implied but not explained; 'reload' is completely ignored. The baseline for low coverage is higher, but the description fails to compensate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (delete) and the resource (dynamic routing rule by ruleid). It distinguishes from sibling tools like drouting_add_rule and drouting_update_rule.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. The purpose is implied by name and context, but no prerequisites, conditions, or cautionary notes are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full burden. It lacks details on pagination behavior (limit/offset), whether list is sorted, or any side effects, leaving the agent uninformed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with no extraneous information, using the minimum words to convey the tool's core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description fails to clarify return format or default behavior; the minimalism leaves significant gaps for a straightforward list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description provides no explanation of parameters (limit, offset) or their meanings, requiring the agent to infer solely from the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List dynamic routing gateways' clearly states a specific verb and resource, and given sibling tools like drouting_list_carriers and drouting_list_rules, it effectively distinguishes its function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, such as drouting_list_carriers or drouting_list_rules, nor are there any use-case exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The word 'Force' implies non-standard behavior, but no details are given about side effects, restrictions, or whether the update is destructive. With no annotations, the description fails to disclose important behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, very concise with no redundant information. However, it may be overly terse, lacking necessary context. Still, it is well-structured and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists, return info is not needed. But for a 'force update' operation, more details are required (e.g., behavior if AOR missing, whether it creates or only updates). The description is too minimal for the task complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter 'aor' is briefly mentioned as 'for an AOR' but lacks format specifics (e.g., SIP URI vs username). Schema coverage is 0%, and the description adds no meaningful details beyond the parameter name.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Force a mid-registrar record update') and identifies the target resource (mid-registrar record for an AOR). It is specific and distinct from sibling tools like mid_registrar_dump.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. The description does not mention context, prerequisites, or when not to use it. A single sentence with no usage hints.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions searching for .xml files but fails to disclose whether the search is recursive, what happens if the directory is invalid, or if it requires read permissions. The behavior is only partially transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise with two sentences that quickly convey the core purpose and parameter. No extraneous information is included, but the brevity sacrifices completeness in other dimensions.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (a simple list of files) and the presence of an output schema, the description does not explain what the returned data contains (e.g., full paths, names). It lacks context on how to interpret results, leaving the agent underinformed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning to the 'scenarios_dir' parameter by stating it is a 'Directory to search for .xml scenario files,' which goes beyond the schema's title and type. However, it does not explain the default value or how the parameter is used (e.g., required or optional).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List available SIPp scenario files,' which is a specific verb and resource. It distinguishes itself from siblings like 'sipp_run_scenario' or 'sip_generate_sipp_scenario' by focusing on listing. However, it does not specify the output format or what constitutes 'available.'
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives. For example, it doesn't suggest using it before running a scenario or warn that it only lists .xml files. The description merely states the action without contextual hints.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description states 'by parsing its PEM certificate' but the input parameter is a domain string, implying the tool fetches the certificate online, which is not transparent. No disclosures about behavior on failure, expired certificates, or authentication needs. Lack of annotations further reduces transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no redundancy. It is front-loaded with the core purpose, but could convey more information within the same length without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple but has an output schema not shown. The description does not mention return values, such as expiry date or status. Given no output schema display, the description should compensate but does not. Also lacks context of prerequisites or related tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% with no parameter descriptions. The description adds minimal context by calling the domain a 'TLS domain' but does not specify format, examples, or constraints. The meaning between 'domain' and 'parsing PEM certificate' is inconsistent.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Check' and resource 'certificate expiry for a TLS domain', clearly distinguishing it from sibling tools like 'tls_check_all_expiry' which checks all domains.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'tls_check_all_expiry'. The description does not provide any when/when-not or alternative recommendations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries all behavioral disclosure burden. It only says 'Update', implying mutation, but does not disclose what happens if domain does not exist, whether it merges or overwrites, or any side effects (e.g., reload). Barely informative.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, 12 words – very concise and front-loaded. However, given the tool's complexity, it is too terse, omitting critical information. Effective structure but insufficient content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description is a stub. It does not explain the many parameters, required input format (e.g., PEM vs path), behavior on existing vs missing domain, or any constraints. Highly incomplete for a 13-parameter mutation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across 13 parameters. The description only hints at 'certificate, key, etc.', leaving 11 other parameters (ca_list, method, cipher_list, etc.) completely unexplained. Fails to add meaning beyond parameter names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Update' and the resource 'existing TLS domain configuration', with parenthetical 'certificate, key, etc.' specifying scope. It distinguishes from sibling tools like tls_add (create) and tls_delete (remove).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., tls_add vs tls_update vs tls_reload). No prerequisites or conditions are mentioned, leaving the agent to infer usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Mentions optional side effect 'triggers MI ds_reload', but otherwise lacks behavioral disclosure beyond what annotations (none) provide. No details on permissions, reversibility, or error handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two efficient sentences with no waste, but front-loading is fine; however, more detail could be added without sacrificing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 11 parameters and a side effect, the description is too minimal. Does not explain return values (output schema exists) or provide sufficient context for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description must add meaning but only references 'by ID'. The 10 optional parameters are completely undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Update a dispatcher destination by ID' with specific verb and resource, and distinguishes from sibling tools like dispatcher_add and dispatcher_remove.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage for updating existing dispatcher destinations, but provides no explicit guidance on when to use vs alternatives or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description only says 'list' with no behavioral details. Since annotations are absent, the description should disclose whether it is read-only, performance implications, or what 'active' means. It does not.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded, and efficient. Could be improved by adding a brief note on scope or safety, but is appropriately sized for its simplicity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Output schema exists, so return values are covered. However, the description lacks context on when to use, what 'active' means, or potential side effects. For a tool with no annotations, this is incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, and schema coverage is 100% (vacuously). Baseline is 4 per rule for zero parameters, though description adds no additional meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'list' and resource 'active TCP and TLS connections'. However, it does not differentiate from sibling tools like 'tls_connections' which may also list connections.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as 'tls_connections' or 'rtpengine_sessions'. The description does not provide context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must carry full behavioral burden. It omits crucial details such as output format, pagination behavior (limit/offset), side effects, or whether the operation is read-only. Only parameter descriptions are given.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is concise with a single-sentence summary and a clear parameter list. Every line adds value, though the parameter formatting could be slightly tighter.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 parameters, no required params, output schema exists), the description adequately addresses core filters but misses pagination details. Output schema likely covers return values, reducing burden, but behavior around offset/limit is unexplained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Description adds meaning beyond schema titles: specifies ISO-format for dates and partial match for caller/callee. However, it does not describe limit and offset parameters, which are present in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description states 'Query missed calls with optional date range and caller/callee filters,' which is a clear verb+resource. However, it does not explicitly distinguish from the sibling acc_query_cdrs, which likely queries completed calls.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like acc_query_cdrs. Missing information about prerequisites or context for querying missed calls.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description mentions the optional triggering of MI address_reload, which is a useful side effect. However, with no annotations, it lacks details on input validation, error handling, or behavior when the entry already exists.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that front-loads the action and includes an important behavioral note about the reload. It is concise with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (8 parameters, no schema coverage, no annotations), the description is far too minimal. It does not explain parameter usage, return value from output schema, or any operational context. The output schema exists but is not referenced.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage and 8 parameters, the description adds no information about any parameter. It fails to explain what each parameter like ip, grp, mask, etc. means or how they affect the operation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Add a new address/permissions entry', specifying the verb and resource. It distinguishes from sibling tools like address_list and address_remove by explicitly indicating creation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, when not to use, or comparisons with other add tools among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that it removes an entry and optionally triggers a reload, but does not disclose side effects, permission requirements, or what happens if the ID is invalid.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence that is front-loaded with the core action, no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple but the description does not cover return values (despite an output schema) or error cases, leaving gaps for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema coverage, the description should explain both parameters fully. It implies id is the identifier and reload triggers reload, but lacks detail on the ID's meaning or reload's effect.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (remove) and the resource (address/permissions entry) via ID, and distinguishes from sibling tools like address_add and address_list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, no mention of prerequisites or when the reload option is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description must cover behavioral traits. It does not disclose whether the operation is read-only, what authorization is needed, or any side effects. It only states it gets parameters, which is minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short and to the point. It front-loads the purpose and lists parameters efficiently. However, it sacrifices detail for brevity, which is acceptable for a simple tool but could be more descriptive.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the large number of sibling cfg tools, the description does not explain how this tool fits into the workflow. It lacks context about output format (though output schema exists) and does not mention related tools like cfg_list_scenarios for listing scenario names.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one parameter with no schema description (coverage 0%). The description adds an example ('load_balancer') and labels it as scenario name. This provides some context but does not explain how to find valid scenario names or that the scenario must exist.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'required and optional parameters for a configuration scenario'. It distinguishes from sibling tools like cfg_list_scenarios (which lists scenarios) and cfg_list_modules (which lists modules).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as cfg_edit or cfg_generate. No mention of prerequisites, context, or exclusions. The description does not help the agent decide when this tool is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description must carry the behavioral burden. It only lists parameters and does not disclose side effects (e.g., whether adding a rule triggers a reload, affects active calls, or requires specific permissions). The 'reload' parameter is mentioned but not explained.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficiently structured with a one-line intro and a bulleted parameter list. It avoids unnecessary words and is easy to scan, though it could be slightly more concise by grouping related parameters.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 11 parameters (4 required), no annotations, and an output schema present, the description is incomplete. It fails to explain several parameters, does not describe return values, error cases, or usage context. The absence of behavioral context further reduces completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning to several parameters (e.g., match_op values, match_exp as pattern, etc.), which is helpful given the schema has no descriptions. However, it omits explanations for match_flags, timerec, disabled, attrs, and reload, leaving 4 of 11 parameters undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Add' and the resource 'dialplan translation rule', making it distinct from sibling tools like delete, update, list, etc. The verb+resource combination is specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., dialplan_update_rule, dialplan_delete_rule). It lacks context on prerequisites, typical use cases, or situations to avoid.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description reveals one side effect (optional MI ds_reload), which is useful. However, without annotations, it omits other behavioral traits such as whether destinations are unique, permissions needed, or what happens on duplicate entries. The disclosure is partial.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—two sentences with no redundant information. Every word adds value: the action, resource, and optional trigger are all included efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (10 parameters, no schema descriptions, no annotations) and the existence of sibling tools, the description is insufficient. It does not cover parameter semantics, prerequisites, or return values, despite having an output schema. The agent would lack critical context for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description provides no additional meaning for any of the 10 parameters. It does not explain what 'destination', 'setid', 'flags', etc. represent, leaving the agent to rely solely on parameter names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Add') and the resource ('new dispatcher destination'), and distinguishes from siblings like 'dispatcher_remove' and 'dispatcher_update' by specifying the operation. The mention of optionally triggering an MI reload adds specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is given on when to use this tool versus alternatives (e.g., dispatcher_update or dispatcher_remove). The description does not mention prerequisites, exclusions, or preferred contexts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description must carry the behavioral burden. It indicates the tool is read-only and checks connectivity and stats, but does not disclose potential performance impact, authorization requirements, or what exactly 'basic runtime statistics' includes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loading the purpose and adding a brief detail. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists (not shown), so the description does not need to explain return values. However, it lacks details on what 'basic runtime statistics' covers, making it minimally complete for a health check tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameters, and schema coverage is 100%. Baseline is 3, and the description adds no parameter information, which is acceptable given zero parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it runs a 'comprehensive health check' against the OpenSIPS instance, checking MI connectivity and retrieving basic runtime statistics. This is specific and distinguishes it from sibling tools that focus on individual statistics or components.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives no guidance on when to use this tool versus alternatives like get_statistics, get_uptime, or opensips_cp_status. It does not mention exclusions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description only says 'retrieve', implying read-only behavior but does not explicitly state that it has no side effects, requires specific permissions, or is rate-limited. It fails to disclose behavioral traits beyond the basic retrieval action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is brief with a clear header and parameter list. It is front-loaded with the core purpose. Could be slightly more efficient by removing the 'Parameters' header redundancy, but overall concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, the description need not cover return values. However, it lacks context on when to use this tool relative to other Homer tools, and does not mention any limitations or constraints. It is minimally viable but not enriched.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 0% coverage (no descriptions in schema). The description compensates by explaining the single parameter: 'The SIP Call-ID to fetch the flow for.' This adds meaningful context beyond the schema's minimal label.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Retrieve the SIP call-flow (ladder diagram) for a call from Homer,' specifying the verb 'retrieve' and the resource 'SIP call-flow'. It distinguishes from sibling tools like homer_get_call_transaction by mentioning 'ladder diagram', though not explicitly contrasting them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like homer_get_call_transaction or homer_search_calls. There is no mention of prerequisites, required state, or scenarios where this tool is preferred.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are absent, so the description must fully disclose behavior. It says 'Force a NOTIFY', implying a write operation, but does not mention side effects, rate limits, or required permissions. No details on the effect on watchers or system state.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is brief and well-structured with a summary line followed by a clean parameter list. Every sentence adds information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, but the description does not explain what the tool returns (success/failure, number of watchers notified). It also lacks system prerequisites or dependencies, leaving the agent without full context for invoking the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description adds significant meaning: it provides an example for pres_uri, lists possible values for event, and explains the numeric refresh_type values. This is valuable beyond the minimal schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Force a NOTIFY to all watchers of a presence URI' clearly states the action and resource. It distinguishes from presence-related siblings like presence_cleanup or presence_list_subscriptions, though it does not explicitly mention alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives like pua_publish or presence_cleanup. No prerequisites or when-not-to-use conditions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It only states the action ('publish') without mentioning side effects, idempotency, or required permissions. The parameter list gives technical details but no behavioral context beyond the obvious mutation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description includes a table of parameters, which is redundant with the input schema but adds value through brief descriptions. It could be more concise by integrating parameter notes into a narrative. It is front-loaded with purpose but then becomes repetitive.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 9 parameters (3 required), no annotations, and an output schema (not shown), the description covers the core parameters adequately but lacks explanation for optional parameters and return value behavior. It is adequate for basic usage but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description lists five core parameters with brief explanations (e.g., 'Presentity URI being published'), adding meaning beyond the schema titles. However, it omits explanations for four optional parameters (id, etag, outbound_proxy, extra_headers) that appear in the schema with defaults. With 0% schema description coverage, the description partially compensates but is incomplete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Publish a presence event via PUA,' which identifies the specific action (publish) and resource (presence event). It distinguishes itself from sibling tools like 'pua_list_publications' and 'presence_refresh_watchers' by focusing on publishing rather than listing or refreshing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., 'pua_list_publications' for viewing, 'presence_refresh_watchers' for subscriptions). It lacks explicit when-to-use, when-not-to-use, or prerequisite conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must disclose all behavior. It only mentions auto-generated HA1 hashes but omits details like idempotency, duplicate handling, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, 10 words, focused and efficient. No superfluous content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 3 params, no annotations, and an output schema not mentioned, the description lacks crucial context about return value (if any), error cases, and what 'auto-generated HA1' implies for security or use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 3 required params with 0% coverage in description. Description hints at password use via hashing but does not explain username or domain semantics beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Create a new SIP subscriber with auto-generated HA1 hashes,' providing a specific verb and resource with a unique distinguishing detail (auto-generated HA1) that sets it apart from sibling subscriber tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like subscriber_get or subscriber_bulk_import. No context about prerequisites or conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description only states that the tool deletes, indicating a destructive action, but no annotations are provided. It omits behavioral details such as whether the deletion is permanent, any side effects, or required permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise at one sentence and front-loaded with the key information. It wastes no words, but could be slightly more informative without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool is simple with one parameter and has an output schema, the description fails to mention what the tool returns or what happens if the subscriber does not exist. It lacks important context for proper invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds minimal meaning to the single parameter 'id' by specifying it is used to identify the subscriber. Since schema description coverage is 0%, this extra context is helpful but still basic.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (delete) and the resource (subscriber) with the identifier method (by ID). It effectively distinguishes itself from sibling tools like subscriber_create, subscriber_get, subscriber_list, and subscriber_update by specifying the delete operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool vs alternatives or any prerequisites. For example, it doesn't mention that the subscriber must exist or that this operation is irreversible.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description must carry behavioral context. It only states the operation without disclosing side effects (e.g., irreversibility, service impact, or confirmation requirements).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that conveys the core purpose without any extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of multiple TLS sibling tools and an output schema, the description lacks details on return value, error conditions, and whether the domain must exist, making it incomplete for an agent to confidently use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter 'domain' has no schema description (0% coverage). The tool description does not elaborate on what 'domain' means (e.g., format, examples, or how to specify it), failing to compensate for the schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Remove') and the resource ('a TLS domain configuration'), which is specific and distinguishes it from sibling tools like tls_add or tls_update.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance provided on when to use this tool vs alternatives, prerequisites, or effects of deletion (e.g., whether the domain must exist, if TLS is reloaded).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behavior. It mentions starting a live trace and optional filters but does not explain the impact (e.g., resource consumption), how results are delivered, or how to end the trace. The output schema exists but adds no behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: one sentence for purpose, then a bulleted list of parameters. No redundant information. The structure is front-loaded with the key action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of starting a live SIP trace, the description is incomplete. It omits details about output format, how to stop the trace, whether it is persistent, and any side effects. The presence of an output schema does not compensate for missing usage context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description compensates by explaining each filter parameter (caller, callee, IP). This adds meaningful semantics beyond the schema's field names and types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool starts a live SIP trace with filters, using a strong verb-noun pair. However, it does not differentiate from sibling tools like trace_control or homer_search_calls, which might serve related but distinct purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus other tracing or monitoring tools (e.g., trace_control, sngrep_capture). The description lacks directives on prerequisites or how to stop the trace.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It only states that it lists entries and optionally filters by group, but does not mention that this is a read-only operation, any authorization requirements, or behaviors like pagination defaults.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with only 10 words, conveying the core function without redundancy or unnecessary information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that the tool has an output schema (not shown), the description is minimally adequate for a simple list tool. However, it omits important details like pagination behavior, default limit, and offset semantics, which are relevant for an agent invoking the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning the schema itself provides no parameter descriptions. The description only hints at the 'grp' parameter for filtering but does not explain 'limit' or 'offset'. More parameter detail is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list' and the resource 'address/permissions entries', distinguishing it from sibling tools like address_add and address_remove. The optional group filtering is also mentioned.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, such as when to use address_add or other list tools. No exclusions or when-not-to-use advice is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description is the only source for behavioral info. It mentions that grouping is computed in Python for cross-backend compatibility, but fails to disclose whether the tool is read-only, has side effects, or requires any permissions. This lack of transparency could lead to misuse.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, using only two sentences plus a parameter list. The key purpose is front-loaded, and every word serves a purpose without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the purpose and parameters adequately given the presence of an output schema. However, it does not specify what the output contains (e.g., fields of the breakdown) or any prerequisites, leaving room for ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning beyond the schema by explaining that 'date_from/date_to' are ISO timestamps and 'group_by' accepts 'hour' or 'day'. It also notes the cross-backend compatibility of the grouping logic, which is not evident from the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it provides a 'Volume and success-code breakdown over a window,' which defines the tool's core function. The tool name 'cdr_summary' reinforces this purpose, and it distinguishes itself from siblings like 'cdr_audit' and 'cdr_count_by_code' by indicating it offers a summarized aggregation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide any guidance on when to use this tool versus alternatives. It only lists parameters, without mentioning scenarios where other CDR tools (e.g., cdr_audit) would be more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that the tool analyzes and suggests optimizations, implying read-only behavior. However, it does not explicitly state that no modifications are made, nor does it describe any side effects, authorization needs, or data handling. Basic transparency is present but not detailed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short (two sentences plus a parameter section) and front-loaded with the main purpose. The parameter section is somewhat redundant but not excessively verbose. Every sentence adds value, though the structure could integrat the parameter description more concisely.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple one-parameter input and the presence of an output schema, the description covers the tool's purpose and input without needing to detail return values. It lists the types of optimizations checked, providing sufficient context for an AI agent to decide when to invoke. However, it omits any mention of prerequisites or limitations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With schema description coverage at 0%, the description adds meaning by specifying that 'config_content' is 'The OpenSIPS configuration text to analyze.' This is a basic explanation but does not elaborate on format, size limits, or specific expectations. For a single required parameter, the added semantics are minimal but adequate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Analyze an OpenSIPS configuration and suggest optimizations,' which is a specific verb+resource. It lists types of issues checked (missing modules, redundant settings, etc.), making the purpose clear. However, it does not explicitly differentiate from sibling tools like cfg_tune, which also focuses on tuning.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as cfg_lint, cfg_validate, or cfg_tune. The description lacks any when/why context, leaving the agent to infer usage without explicit direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so description must fully disclose behavior. It mentions deletion and optional reload but does not cover safety aspects (e.g., what if ID not found), permissions, or whether operation is destructive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, two concise clauses, no unnecessary information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Output schema exists but description does not explain return value. Lacks details on edge cases, errors, or behavioral nuances beyond basic operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must compensate. It adds context that 'id' identifies the destination and 'reload' triggers a reload, but does not explain what a dispatcher destination is or what the reload entails.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action 'Remove' and the resource 'dispatcher destination', and distinguishes from sibling tools like dispatcher_add and dispatcher_update.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance provided on when to use this tool vs alternatives (e.g., dispatcher_add, dispatcher_update), nor any prerequisites or scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose side effects (e.g., overwriting current rules), required permissions, or whether the operation is reversible. It only states the action and data source.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with the action, no redundant words. Ideal conciseness for a simple tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description lacks usage guidance and behavioral context. For a mutation tool with no annotations, it is insufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters with 100% coverage, so the description does not need to explain parameters. Baseline 3 applies as per guidelines.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool reloads LCR rules and gateways from the database, using a specific verb and resource. It distinguishes from sibling reload tools (e.g., carrierroute_reload) by specifying 'LCR'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives like lcr_dump or carrierroute_reload. The agent receives no context about prerequisites or appropriate scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It mentions inserting a handler and updating an array but does not discuss side effects (e.g., ordering, impact on existing code, error handling, permissions). Minimal transparency for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with four sentences. The first sentence states the main purpose, followed by a brief explanation of effects, then a clear parameter list. No wasted words; essential information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has an output schema (per context signals) but the description does not mention return values, error conditions, or validation of inputs. It assumes prior knowledge of OpenSIPS module structure and does not specify what 'existing module source' means in terms of format or completeness. Incomplete for a code-generation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% per signals, but the description lists each parameter and provides a brief explanation. For command_name, an example is given. However, explanations are somewhat tautological (e.g., 'Description of what the MI command does') and do not add constraints or formats beyond the schema. Adequate but not enriched.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Add an MI command'), the resource ('existing OpenSIPS module C source'), and what it does internally ('Inserts the MI command handler function and updates the mi_cmds array'). This distinguishes it from sibling tools like 'module_add_param' by specifying the target code construct.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives (e.g., module_add_param, module_compile). No prerequisites or context about when it's appropriate to add an MI command. The description omits any usage hints.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility. It states the tool adds a parameter and updates the params array, but does not clarify whether the operation modifies the source in place or returns a new version. It does not disclose any side effects, authorization needs, or whether the tool is destructive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short and to the point, with a clear opening sentence followed by a structured parameter list. It avoids unnecessary words, though the parameter list could be more integrated into the prose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 4 parameters with 0% schema coverage and has an output schema. The description explains what the tool does and gives parameter summaries but does not describe return values or how the output is structured. Given the complexity of modifying C source code, more detail about the expected output would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must add meaning. It lists parameters with minimal explanation: param_type exemplified as 'str or int', default_value described as 'Default value', and param_name given an example 'db_url'. This provides some semantic value beyond the bare schema, but details like format constraints or allowed values are missing.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Add' and the resource 'module parameter to an existing OpenSIPS module C source'. It specifies the actions performed: inserts the parameter variable declaration and updates the params array. This distinguishes it from sibling tools like module_add_mi_command or module_compile.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide any guidance on when to use this tool versus alternatives. There is no mention of prerequisites, such as ensuring the module source exists, or when not to use it. No alternatives are suggested.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should fully disclose behavior. It only states 'List active RTPEngine media sessions' without mentioning side effects, authentication needs, or performance considerations. The read-only nature is implied but not explicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no wasted words, perfectly front-loaded with the essential purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (not shown), the description does not need to detail return values. However, it lacks guidance on scope or performance, which could aid an agent in deciding when to use this tool. It is minimally complete for a parameterless listing tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so baseline score is 4. The description implicitly conveys that all active sessions are listed without filters, which is adequate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('List') and resource ('active RTPEngine media sessions'), clearly indicating the tool's function. It distinguishes itself from sibling tools like 'rtpengine_terminate_call' which perform different actions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, nor any exclusions or prerequisites. The description lacks context for appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility. It only mentions running a load test, but lacks details on side effects, required permissions, or any destructive potential. This is insufficient for a load testing tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, front-loaded with the core purpose, and contains no extraneous information. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists, the description could focus on usage behavior. However, it omits details like return values, behavior under load, or how results are provided. For a load test tool, this is incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning parameter titles are the only documentation. The description does not explain any of the five parameters (target, cps, duration_sec, out_dir, label) beyond mentioning TARGET. This fails to compensate for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool runs a SIPp INVITE load test against a target. It also distinguishes from the sibling tool bench_quick_register by specifying that this exercises the call path rather than the registration path.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context by comparing to bench_quick_register, implying that this tool is for call path testing while the sibling is for registration. However, it does not explicitly state prerequisites or when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description must cover behavior. It describes the tool as a report combining multiple commands, suggesting read-only intent. But it lacks details on permissions, side effects, or performance.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise, front-loaded with the purpose, and every sentence adds value. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although an output schema exists, the description partially covers what the output contains (capabilities, sharing tags, topology). However, it does not describe the input parameter, and the overall completeness is adequate but not thorough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one required parameter (cluster_id) with 0% description coverage. The tool description does not explain what cluster_id represents, relying solely on the name and context. This is a significant gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it reports cluster sync health combining capabilities, sharing tags, and topology. It includes the operator goal of checking replication health, but does not explicitly differentiate from sibling tools like cluster_status or cluster_topology.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies it should be used as a single call to replace three separate clusterer_list tools, giving context for when to use it. However, it does not specify when not to use it or mention alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must carry full burden. It only states 'recent logs' without clarifying behavior like whether logs are continuous, if it requires a running container, or any cost implications. Read-only nature is implied but not explicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is very concise with two sentences and a clear parameter list. Every sentence is necessary and no extraneous information. Structure is front-loaded with the main purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple log retrieval tool, the description is adequate but could mention output format or that logs are textual. With output schema present, the burden is reduced, but some context about what 'recent' means is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description compensates slightly by providing defaults and short descriptions for service and lines. However, it lacks explanation of valid values or error handling, which would enhance parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Retrieve recent logs from a Docker Compose service' which is a specific verb+resource. It distinguishes from siblings like docker_status, docker_restart, and others focused on deployment or status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool instead of alternatives. There is no mention of prerequisites, context for log retrieval, or comparison with other logging tools among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully convey behavioral traits. It only states the tool wraps an API query returning correlated IDs. It does not disclose read-only nature, authentication requirements, rate limits, or error behavior. The behavioral transparency is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences (approximately 50 words) with no filler. It front-loads the main purpose and includes necessary technical context without excess. It is efficiently written.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 3 required parameters and no annotations, the description leaves gaps: it does not explain the time range parameters, nor any prerequisites or behavioral traits. Although an output schema exists, the input context is incomplete for a correct tool invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, yet the description only explains the 'callid' parameter implicitly. It fails to explain 'from_ts' and 'to_ts' which define the time range for correlation. The description adds minimal meaning beyond the parameter names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: correlating a Call-ID across OpenSIPS hops. It uses specific verbs ('correlate') and resource ('Call-ID'), and distinguishes from sibling tools by focusing on correlation across hops. No tautology.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool (when topology_hiding or B2BUA is present, to find sibling Call-IDs). However, it does not explicitly state when not to use it or mention alternative tools like homer_search_calls for basic call search. The guidance is present but not comprehensive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the behavioral burden. It only states the action without disclosing side effects, permissions, or the meaning of 'active'. Minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no extraneous words. Information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and an output schema, the description provides the essential purpose. However, it lacks broader context about when to use this tool among many list tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so schema coverage is 100%. Baseline is 4 for zero parameters; description adds no additional param info, but none is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List active PUA publications' with a specific verb and resource. The parenthetical '(OpenSIPS-as-publishing-UA)' adds context but does not explicitly distinguish from sibling tools like presence_list_subscriptions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as pua_publish or presence_list_subscriptions. The description lacks when-not context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only states the action (login/logout) without disclosing side effects, idempotency, prerequisites, or response behavior. The description does not contradict annotations as none exist.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is succinct (one sentence plus parameter list) and front-loaded with the core action. It avoids redundancy but could be slightly more structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity and presence of an output schema, the description adequately covers the parameters. However, missing context on return values, side effects, or prerequisites makes it only minimally complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, so the tool description compensates well by providing clear semantics: agent_id as an identifier example, and state as numeric 1/0 for login/logout. This adds meaning beyond the schema's titles.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Log a call center agent in or out,' clearly defining the verb and resource. It distinguishes from sibling tools like cc_list_agents or cc_list_queue, which focus on listing rather than changing agent state.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, no prerequisites, and no scenarios where login vs logout is appropriate. Only basic parameter values provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden of behavioral disclosure. It mentions the tool is cheap (low cost), which is useful. But it does not state whether it is read-only, any required permissions, or other side effects. The presence of an output schema reduces the need to describe return values, but other behavioral aspects are missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two short sentences. The first states the core function, and the second adds context and value. No redundant or unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple aggregation tool with two parameters and an output schema, the description covers the purpose, use case, and performance characteristic adequately. It does not explain the output format, but that is handled by the output schema. Slight lack of guidance on parameter details prevents a perfect score.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description should compensate by explaining parameters. The description mentions 'within a window' which loosely relates to date_from and date_to, but it does not describe the format, constraints, or how they are used. Most of the parameter meaning must be inferred from names alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it counts CDRs by SIP response code within a date window, and gives a concrete use case ('what's the 5xx rate right now?'). It distinguishes itself from siblings only implicitly by mentioning 'SQL-side aggregation' and being cheap, but does not explicitly differentiate from related tools like cdr_summary or cdr_audit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description suggests using it for a quick-glance aggregate report, implying a lightweight use case. However, it provides no explicit guidance on when not to use it or what alternatives (e.g., cdr_summary, acc_query_cdrs) are better suited for other scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. While it describes the input/output flow, it does not clarify whether starting a session has side effects (e.g., state creation), auth requirements, or rate limits. It also fails to mention if the tool is read-only or destructive, which is critical for agent decision-making.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with a few sentences, clearly structured with a brief summary and a parameter section. It avoids unnecessary details, though the parameter list could be more integrated. Overall, it is efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (one parameter, output schema exists), the description is sufficiently complete. It explains the input, output elements, and the session-starting purpose. The presence of an output schema reduces the need to describe return values, and the description adds value for the single parameter.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has only one parameter with 0% description coverage, so the description must document it. It adds value by specifying that 'description' is a 'natural-language description of the deployment the user wants', which is more informative than the schema's title 'Description'. This aids the agent in proper use.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Start an interactive m4 config-building session' with a clear verb and resource. It explains the input (NL description) and output (scenario, parameters, next-step). It implicitly distinguishes from sibling tools like cfg_edit or cfg_generate by emphasizing the interactive session start, but does not explicitly differentiate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when the user wants to start a config session from natural language. However, it does not provide explicit guidance on when not to use this tool, nor does it mention alternatives among the many cfg_* siblings. It lacks exclusions or context for optimal use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden for behavioral disclosure. It implies a non-destructive read operation via the verb 'List', but does not explicitly state that it is read-only, any authentication requirements, or side effects. It is minimally adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence of 10 words, front-loaded with the main action. Every word is necessary, no redundancy. Excellent conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the core purpose and the optional filter, but omits mention of pagination (limit/offset) and does not explain the output schema. Given the tool's simplicity and presence of an output schema, it is minimally complete but could be improved with notes on limit/offset defaults.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description only hints at the 'dpid' parameter ('optionally filtered by dialplan id') but does not explain 'limit' or 'offset'. No additional meaning is added beyond the schema, failing to compensate for the lack of descriptions in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'dialplan rules', and mentions optional filtering by dialplan id. This exactly identifies the tool's function and distinguishes it from sibling tools like add, delete, or reload.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no explicit guidance on when to use this tool versus alternatives. It does not mention prerequisites, when not to use it, or compare with similar list tools like drouting_list_rules or dialplan_translate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description discloses subscription behavior including permanent subscription via expire=0 and socket format details. However, it does not discuss potential side effects (e.g., resource consumption) or whether subscriptions persist beyond OpenSIPS restart for non-zero TTL.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a title line followed by a bulleted list. It is concise and front-loaded, containing only essential information. Minor verbosity in socket format examples could be trimmed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (not shown) and the complexity of subscription tools, the description covers parameters adequately. It lacks information about return values or errors, but the output schema likely addresses that. Overall, it provides sufficient context for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 0% schema description coverage, the description thoroughly explains all three parameters. It provides concrete examples for event format, socket backend formats (udp, rabbitmq, flatstore), and the special meaning of expire=0. This adds significant value over the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Subscribe to' and the resource 'an OpenSIPS event at a given transport socket'. It explains the purpose but does not explicitly distinguish from sibling tools like 'event_list' or 'event_list_subscribers', though the name suggests a write operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. The description lacks context about typical use cases, prerequisites, or scenarios where other tools would be more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must disclose behavioral traits. It only states listing with optional filter, omitting whether the operation is read-only, requires special permissions, or returns all rules when no filter is applied. Minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, concise and front-loaded. No unnecessary words. Efficiently communicates core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (list with optional filter) and the presence of an output schema, the description is largely complete. It could mention default behavior (returns all rules if no profile ID), but the output schema covers return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description must compensate. It adds meaning to 'profileid' by noting optional filtering. However, it does not explain 'limit' and 'offset' (though pagination is implied). Partial compensation, but not exhaustive.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states action ('List') and resource ('fraud detection rules'), with optional filter by profile ID. Distinguishes from sibling fraud tools (add, delete, update, reload) by specifying the listing operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus other list tools (e.g., drouting_list_rules, dialplan_list_rules) or the other fraud tools. Lacks any when-not-to-use or alternative recommendations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description states the action but omits behavioral details like side effects (e.g., files created, time taken), required environment (build tools), or failure modes. Without annotations, the description should compensate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is reasonably concise with a clear purpose statement and parameter explanations. However, the parameter list format is slightly verbose; could be more compact.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description explains the core function but omits details about output (e.g., success/failure messages) and required environment. Since an output schema exists, it partially compensates, but completeness could be improved.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning beyond the schema by describing module_name as 'Name of the module directory under modules/' and source_path as 'Path to the OpenSIPS source tree root' with a default. This clarifies the parameters effectively despite 0% schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Compile') and resource ('OpenSIPS module from source'), and explains it runs `make modules` for the specified module. It distinguishes well from sibling tools like module_add_mi_command or module_validate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives, prerequisites (e.g., source code must be present), or typical use cases. The description only states the parameters without context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only states that it 'Produces a markdown README' but does not disclose whether it overwrites existing files, requires a specific module structure, or if it has any side effects. Behavioral traits like read-only vs destructive are not addressed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise: a one-sentence title, a brief paragraph, and a bullet list for parameters. It front-loads the purpose and uses minimal words. Every sentence contributes value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the description need not detail return values. It mentions the output is markdown, which suffices. It covers the required parameter (name) and describes optional parameters. However, it does not explain the exact structure of the JSON arrays or mention if the module must already exist, but overall it is sufficiently complete for a documentation generation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must add meaning. The bullet list provides brief definitions for each parameter, e.g., 'Module name.' and 'JSON array of MI commands (same format as module_scaffold).' This adds some context but lacks detail on exact format or constraints. It references another tool for format, which is helpful but not fully self-contained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Generate README documentation for an OpenSIPS module.' It further explains the output is a markdown README with parameter, MI command, and usage documentation. This distinguishes it from sibling tools like module_scaffold (which creates module structure) and module_validate (which validates modules).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It does not mention prerequisites (e.g., if a module must already exist) or scenarios where generating docs is appropriate. The description is purely functional without contextual advice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It states it generates a scrape configuration but does not disclose output format, return behavior, side effects, or whether it creates files. The existence of an output schema mitigates this slightly, but the description alone is insufficient for behavioral clarity.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear first sentence and organized parameter list. It is concise but could be slightly more compact by avoiding the docstring format. Slightly verbose for the focused task.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the 6 parameters and no annotations, the description provides a coherent understanding of the tool's operation, especially the target generation logic. It does not describe the output structure, but the output schema likely covers that. Overall, it is nearly complete for an agent to use effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description adds substantial meaning: deploment_name's role as instance label, default target logic, metrics_path default, and targets list behavior. This goes well beyond what the schema provides, fully compensating for the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Generate a Prometheus scrape configuration', which is a specific verb and resource. However, it does not explicitly differentiate from sibling observability tools like 'observability_inject_prometheus' or 'observability_generate_bundle', which could have overlapping functionality.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains parameter roles but provides no guidance on when to use this tool versus alternatives, no prerequisites, and no use-case scenarios. It is purely parameter-focused without contextual usage advice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description only states the action. Lacks disclosure on permissions, idempotency, or effects on other systems. Does not mention what happens if IP is not in the block list.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise at two sentences plus parameter list. Front-loaded with action, no redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Lacks information about return value or output schema, error handling, and idempotency. Despite having an output schema, its content is not described. Insufficient for a production tool given the complexity of sibling tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage for parameter 'ip'. Description explicitly states 'The IP address to unblock', adding meaning beyond the schema's type 'string' and title 'Ip'. However, does not specify format (e.g., IPv4/IPv6, CIDR).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states action: 'Remove an IP address from the Pike block list.' Verb 'Remove' and resource 'IP address' are specific, distinguishing it from similar tools like pike_list_blocked.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives like fail2ban_unban_ip or address_remove. No context on prerequisites or expected scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description does not disclose side effects, error handling, or what happens if the contact does not exist. It only describes the basic removal action without behavioral context beyond the obvious destructive nature.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with a one-line purpose and a structured Parameters section. However, it could be slightly more compact by removing redundant wording.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple with 3 required parameters and no output schema. The description covers what the tool does and parameter examples, but it does not mention return values or success/failure behavior. Sibling tools are not referenced, leaving some context missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description adds significant value by providing examples for each parameter (e.g., 'table: The usrloc table (e.g. location)'). This helps the agent understand the expected format and values beyond the plain schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Remove a specific contact binding from the user location table' with a specific verb and resource. Examples for each parameter further clarify the purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives like ul_add_contact or ul_dump. No mention of prerequisites or conditions, leaving the agent without contextual decision support.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the burden. It discloses the action (terminate) but lacks details on side effects, authority needed, error handling, or idempotency. Critical behavioral traits for a termination action are missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: a one-line purpose statement and a one-line parameter description. No redundant information, front-loaded, efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool is simple with one parameter and an output schema exists, the description is adequate but not fully complete. It lacks information about behavioral expectations (e.g., what happens on success/failure, permissions needed), which is important for a termination action.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has no description for the parameter (0% coverage). The description adds value by explaining that the key is 'as reported by b2b_list_sessions', providing a clear source for valid values. This goes beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'terminate' and the resource 'B2B session', and distinguishes from sibling tools like b2b_bridge and b2b_trigger_scenario which have different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives, nor any exclusions. It only notes the prerequisite source for the key (b2b_list_sessions), but no comparison to other termination methods or conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the full burden. It does not disclose whether the tool is destructive, read-only, or requires specific permissions. The behavioral impact of triggering a scenario (e.g., starting a call, modifying state) is unexplained.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—one sentence for the purpose followed by a clear, structured parameter list. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 3 parameters (1 required, 2 with defaults) and an output schema presumably indicating success/failure, the description covers the core purpose but omits expected behavior like prerequisites (scenario must exist), error handling, or return value details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite the schema having no descriptions (0% coverage), the description adds meaningful context: scenario_id is from b2b_logic XML, custom_hdrs are optional SIP headers, and params are semicolon-delimited per-scenario. This compensates for the schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it triggers a B2B scenario by ID, referencing XML scenarios. The verb 'trigger' and resource 'B2B scenario' are specific, and it distinguishes from siblings like b2b_bridge or b2b_terminate by its focus on scenario invocation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as sipp_run_scenario or b2b_bridge. There are no explicit conditions or exclusions mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the description does not disclose behavioral traits such as potential locking, duration, or side effects of reloading.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded, no unnecessary words. Perfectly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and output schema present, the description is adequately minimal but lacks context about when or why to reload, and any behavioral impacts.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has zero parameters, achieving 100% coverage. Baseline for no parameters is 4; description adds no further parameter info, which is fine.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clear verb+resource: 'Reload carrier route tables from the database.' Distinguishes from sibling tools like carrierroute_activate_host, carrierroute_dump, etc. which perform different operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives. Does not specify prerequisites or typical use cases (e.g., after database modifications).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description does not disclose side effects (e.g., whether it modifies state, requires specific permissions, or is resource-intensive). The command is explained but behavioral traits are missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: two sentences and a parameter list. No redundant information. Every part serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that an output schema exists, the description does not need to detail return values. However, it lacks information about expected input format, error behavior, or constraints. Adequate but not thorough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Parameter 'config_content' is described as 'The full OpenSIPS configuration text to validate,' adding meaning beyond the schema (which only gives type and title). Schema coverage is 0%, so this description compensates well.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'Validate' and resource 'OpenSIPS configuration', and specifies the exact command used. Distinct from sibling tools like cfg_check_compat or cfg_dry_run.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., cfg_check_compat, cfg_dry_run). Does not mention prerequisites or context for invocation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavioral traits. It only states 'Get' (implying read-only) but does not explicitly confirm no side effects, rate limits, or permissions. Lacks sufficient detail for a safe invocation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, with only two short sentences that are front-loaded and efficiently convey the core functionality. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema, the description need not detail return values, but it still lacks explanation of what 'dialog profile information' includes beyond size. It is minimally adequate but could be more informative.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds significant meaning beyond the input schema: it explains that the profile parameter queries size for a specific name, and omitting it lists all profiles. Schema had 0% description coverage, so this compensates well.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'dialog profile information', and explains the behavior when the optional parameter is omitted versus provided. It distinguishes from sibling tools like dlg_list (lists dialogs), dlg_end, and dlg_push_var.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no explicit guidance on when to use this tool over sibling tools (e.g., dlg_list, dlg_end). It only describes the parameter, leaving the agent to infer usage context without clear selection criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears the full burden of behavioral disclosure. It does not mention that deletion is permanent, whether it checks for rule existence, or any side effects. The minimal description falls short for a destructive operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. Every word serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete operation, the description covers the basic action and parameter. However, it could be improved by noting that deletion is irreversible or that the rule must exist. Given the presence of an output schema, the return value is not required in the description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter 'ruleid' is documented in the schema only with type and title. The description adds value by clarifying that it is used to specify the rule to delete ('by rule ID'), which compensates for the 0% schema description coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (Delete) and the resource (fraud detection rule) and specifies the method (by rule ID). This distinguishes it from sibling tools like fraud_add_rule, fraud_list_rules, and fraud_update_rule.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like fraud_update_rule or fraud_list_rules. The description only implies usage for deletion but provides no exclusions or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description must disclose behaviors. It only states 'reload' without explaining side effects, required permissions, or whether it interrupts active fraud detection. The description is insufficient for understanding impact.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, front-loaded with the action 'reload' and the resource, with no superfluous words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless tool with an output schema (not detailed), the description is minimal but adequate. However, it omits what triggers a reload, prerequisites, or return value content, leaving some gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so parameter semantics are not needed. The baseline is 4, and the description adds context by mentioning 'via MI command', which clarifies the mechanism.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description specifies the verb 'Reload' and the resource 'fraud detection rules' in OpenSIPS via MI command, clearly distinguishing it from sibling tools like fraud_add_rule or fraud_list_rules.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, such as when rules need updating or after configuration changes. There are no exclusions or context indicators.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It only says 'Show,' implying read-only, but does not disclose side effects, permissions, or whether data is instantaneous or aggregated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded, no fluff. Every word adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and an output schema, description is minimally adequate. Could specify scope (e.g., per process vs system-wide) to improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so baseline 4 applies. Description adds no param info, which is acceptable since schema coverage is 100%.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool shows OpenSIPS memory usage details for shared and private memory, using specific verb and resource. Distinguishes from siblings by focusing on overall memory usage rather than specific performance metrics.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs other memory/perf tools like perf_memory_report. Lacks context on prerequisites or typical use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, and the description does not disclose behavioral traits such as whether the reload disrupts active connections, requires special permissions, or is safe to call repeatedly.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, clear sentence that efficiently conveys the tool's purpose without unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless tool with an output schema, the description is minimally adequate. However, it lacks details on effects (e.g., hot reload vs. requiring restart) and could benefit from brief behavioral context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, and schema coverage is 100%, so the baseline score is 4. The description implies no arguments are needed, though it does not explicitly state this.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Reload') and the resource ('load balancer destinations from the database'), distinguishing it from other reload tools like carrierroute_reload or tls_reload.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, nor any prerequisites or typical use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries full burden. It states 'dump' which implies a read operation, but does not explicitly confirm read-only, idempotency, or lack of side effects. The description lacks behavioral disclosure beyond the basic action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that immediately conveys the purpose. It is front-loaded with the verb and resource, containing no extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity (no parameters, an output schema exists), the description is adequate but minimal. It does not explain what the output contains or whether the dump is safe. It could be more complete by mentioning 'read-only' or the nature of mappings.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so the input schema fully covers the interface. With zero parameters, the description has no additional semantic load to carry. Baseline is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Dump', the resource 'mid-registrar AOR state', and specifies the mapping direction 'UAC→upstream'. This is distinct from sibling tools like mid_registrar_update and other dumps (e.g., ul_dump, lcr_dump).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like mid_registrar_update or other dump tools. There is no mention of use cases or exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions parsing 'loadmodule' directives and emitting dashboard JSONs, but does not explicitly state whether the tool modifies any state, requires specific permissions, or has side effects. For a generation tool, it is reasonable to assume it is read-only, but this is not confirmed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: a clear one-sentence summary, a paragraph on input quality, and a bulleted list of parameters. Every sentence adds value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the description does not need to detail return values. It explains the tool's core functionality, input constraints, and high-level output ('emits one dashboard JSON per triggered panel category'). This is sufficient for an agent to understand the tool's role.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description compensates well by detailing each parameter: config_content is 'Full OpenSIPS configuration text', deployment_name is 'Slug used in dashboard titles and filenames', and extra_tags is 'Additional Grafana tags appended to every dashboard.' This adds significant meaning beyond the schema's type and title.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Synthesize Grafana dashboards from any OpenSIPS configuration text.' It specifies the resource (OpenSIPS cfg) and the action (synthesize dashboards). However, it does not explicitly differentiate from sibling tools like observability_generate_dashboards or observability_generate_bundle_from_cfg, which could cause confusion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides context on what inputs are valid: 'Works on hand-edited configs, the output of cfg_generate, the output of cfg_build_from_flags — anything that's syntactically a valid OpenSIPS cfg.' This helps the agent know when to use it, but there is no guidance on when NOT to use it or how it compares to alternatives like observability_generate_dashboards.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavioral traits. It only states the reset action without mentioning side effects, reversibility, permissions, or impact on ongoing operations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences and a parameter list. Every part is necessary and front-loaded with the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is adequate for a simple reset tool but fails to mention the return value or any confirmation of success. Given the tool has an output schema (per context), the description could be more complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% coverage for the 'pipe' parameter, but the description explains that it is 'the name of the pipe to reset,' adding meaningful context beyond the schema's raw type definition.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Reset a rate limiting pipe counter to zero,' using a specific verb and resource. It distinguishes this tool from its sibling 'ratelimit_status' which likely shows status rather than resets.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, nor any conditions or prerequisites for resetting a pipe. The agent receives no context for appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden but only states the basic action. It does not disclose potential side effects (e.g., whether it disrupts active calls) or prerequisites (e.g., database connectivity). Minimal behavioral info beyond the obvious.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, focused sentence with no wasted words. Front-loads the key action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple reload with no params and an output schema, but lacks any mention of impact or context. In a large sibling group, this could be insufficient for an agent to fully understand when to invoke it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so schema coverage is 100%. The description adds no parameter info but is unnecessary. Baseline 4 is appropriate for zero-param tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Reload') and the resource ('RTPEngine configuration from the database'). It is specific and distinguishes from sibling tools like carrierroute_reload or rtpengine_enable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like rtpengine_enable, rtpengine_show, or other reload tools. The agent lacks context for appropriate selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided. The description does not disclose behavioral traits such as authentication requirements, rate limits, or any side effects. Since it is a read-only command, the minimal description is insufficient to convey safety or performance implications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise. However, it is extremely brief; a bit more context (e.g., that it returns a list of instances with status) would improve completeness without sacrificing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has an output schema (present but not shown). Given the description and the presence of an output schema, the description adequately covers the purpose. However, considering the complexity of RTPEngine, additional context about the output format might be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so schema_description_coverage is 100%. With no parameters, the description does not need to add parameter-specific details. The baseline is 4, and the description adds no value but also does not detract.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Show' and the resource 'all RTPEngine instances and their current status.' It is specific and distinguishes from sibling tools like rtpengine_enable, rtpengine_reload, and rtpengine_sessions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. For example, it does not mention that rtpengine_stats provides aggregated metrics or that rtpengine_instance_health checks individual instance health.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description lacks any behavioral details such as idempotency, authentication requirements, or potential side effects. With no annotations provided, the description carries full responsibility and fails to disclose that this is a read-only operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, succinct sentence that immediately conveys the tool's function. It is front-loaded with the key action and resource, with no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is minimal but acceptable given the existence of an output schema that likely details the statistics returned. However, it does not specify the scope of statistics (e.g., system-wide or per call), leaving some ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, and the description adds no parameter information beyond what the schema provides. Since schema coverage is 100%, a baseline of 4 is appropriate as no additional parameter semantics are needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Retrieve' and the resource 'RTPEngine media proxy statistics', making the tool's purpose unambiguous. It distinguishes itself from sibling tools like rtpengine_enable or rtpengine_sessions by focusing exclusively on statistics retrieval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when or when not to use this tool versus siblings like rtpengine_show or rtpengine_sessions. Given the many rtpengine-related tools, the lack of contextual usage recommendations is a significant gap.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are absent, so the description must disclose behavioral traits. It only states a read operation ('Get') but does not mention error handling, authentication requirements, rate limits, or what happens if the subscriber is not found. The description provides minimal behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, focused sentence with no redundant information. It efficiently conveys the core action and required parameters.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read operation with an output schema available, the description suffices in explaining the tool's primary function. However, it could benefit from mentioning that it expects exact matches and does not support partial searches, and it lacks any note on behavior when parameters are malformed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. The description adds minimal value by saying 'by username and domain', which implicitly ties the parameters to the identification fields. However, it does not provide format, constraints, or examples, leaving significant ambiguity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'single subscriber', and specifies the identifying parameters 'by username and domain'. This distinguishes it from sibling tools that create, list, update, or delete subscribers.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for fetching a specific subscriber with known credentials, but it does not provide explicit guidance on when to use this tool versus alternatives like subscriber_list or subscriber_search. No when-not-to-use or alternative tool mentions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of disclosing behavior. It only says 'reload,' which implies a potentially state-changing operation, but does not clarify side effects (e.g., whether existing connections are dropped, if certificates are re-read), permissions needed, or whether it is non-disruptive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 8 words, perfectly concise and front-loaded with the verb. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and the presence of an output schema, the description provides sufficient context for a simple reload command. However, it lacks information about the output (though schema covers that) and could elaborate on what 'reload' entails (e.g., effect on current TLS sessions).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, and schema coverage is 100%. The description adds no parameter details, which is appropriate given the absence of parameters. Baseline for 0 parameters is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (reload) and resource (TLS domains), and context (OpenSIPS via MI command). It effectively distinguishes itself from sibling tools like tls_add, tls_delete, tls_list, and tls_update by focusing on the reload operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, nor any prerequisites or context. The description simply states what it does without explaining scenarios (e.g., after configuration changes) or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It informs that the tool shows mapped keys for header rewriting, implying a read-only query. However, it does not explicitly state read-only nature, performance implications, or any side effects, leaving gaps in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences. The first sentence delivers the core purpose, and the second adds relevant detail. Every word is necessary, and no superfluous content is present. Front-loaded effectively.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and an output schema exists, the description explains what is returned (mapped keys) but omits context such as authentication needs, frequency of use, or potential impacts. It is adequate minimally but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, and schema description coverage is 100%. The baseline is 3, and the description adds no parameter information because none exist. No additional value is provided beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists active dialogs with topology hiding engaged, which is a specific verb and resource. It also explains what it shows (mapped keys for header rewriting), making the purpose unambiguous and distinct from sibling tools like dlg_list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., dlg_list for all dialogs), nor any preconditions or exclusions. It simply states what the tool does without context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description does not disclose behavioral traits such as whether starting a trace overrides existing ones, if it requires specific permissions, or if it has any side effects. The tool's effect on ongoing traces is unclear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loads the purpose. The parameter list is clear but could be more integrated into a single narrative. However, it avoids unnecessary detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's control nature and the presence of an output schema, the description sufficiently covers the actions and parameter requirements. It does not explain return values, but the output schema likely covers that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds significant meaning beyond the schema by explaining that 'action' accepts start, stop, or status, and that trace_uri is required only for start. This compensates for the schema's 0% description coverage and lack of enums.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool controls SIP tracing with start, stop, or status actions. It specifies the resource (SIP tracing) and the verb (control), and distinguishes from siblings like trace_start_live by defining precise actions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like trace_start_live or other tracing tools. The description does not provide context for selecting this tool over siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description must carry behavioral transparency. It only states 'reload from database' without mentioning side effects (e.g., impact on active registrations, safety profile, or whether it triggers re-registration).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is concise, front-loaded with the action, and contains no extraneous words. Every word is necessary.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple action and absence of parameters, the description is moderately complete. However, it lacks details about return values (despite an output schema existing) and potential side effects, which are important for a reload command.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so the baseline is 4. The description successfully conveys the tool's purpose without needing parameter details. No additional semantic value is required.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Reload') and the resource ('UAC registrant configuration from the database'). It distinguishes from sibling tools like uac_reg_list (list) and uac_reg_enable (state change).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage context is provided. There is no guidance on when to use this tool versus other reload or registration tools, such as after configuration changes or to refresh state.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It describes a read operation with optional filters but does not mention side effects, pagination, or data scope. It is adequate for a query tool but lacks detail on return format and limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise: a single-sentence purpose followed by a clear parameter list. No wasted words, well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that an output schema exists (so return format is defined there), the description covers the tool's purpose and main filter parameters. However, it omits pagination parameters (limit, offset) which could impact invocation. Overall, it provides enough context for correct use in most scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description covers 4 of 6 parameters (date_from, date_to, caller, callee) with format examples (ISO for dates, partial match for text). However, limit and offset are not described in the description, despite their presence in the schema. The description adds value beyond the schema but is incomplete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Query', the resource 'accounting CDRs', and specifies optional filters. It distinguishes itself from the sibling 'acc_query_missed' by focusing on general CDRs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives like 'acc_query_missed'. The description only covers what it does, not when to choose it over siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It only states the tool lists entities, with no disclosure of behavioral traits such as read-only nature, authentication requirements, rate limits, or side effects. It is insufficient for a tool with no annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that conveys the core functionality without any superfluous words. It earns its place efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema and no parameters, the description sufficiently explains the tool's purpose. It could be slightly more precise (e.g., clarifying 'raw'), but overall it is complete for a list operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0 parameters with 100% coverage, so the description adds no parameter info. Baseline is 3 per schema coverage rule; the description does not need to add more since no parameters exist.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'raw B2B entities (UAC/UAS/server)', explicitly differentiating from sibling tools like b2b_list_sessions which list sessions. It is specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for listing entities tracked by b2b_entities, but provides no explicit guidance on when to use this tool vs. alternatives (e.g., b2b_list_sessions, b2b_bridge). No when-not-to-use or alternative references are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must convey behavioral traits. It only says 'Dump' which implies a read operation, but it does not explicitly state that it is non-destructive, requires no permissions, or has no side effects. The agent is left to infer safety.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 9 words, front-loaded with the action and resource, with no unnecessary information. It is highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values are covered. The description is adequate for a simple dump tool, but it lacks context such as whether the dump is of the current runtime state or if it triggers any internal events.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters (0 params), so baseline is 4. The description adds meaning by specifying the content of the dump (carriers, domains, prefixes, hosts), which is helpful beyond the empty schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Dump loaded carrier route tables (carriers, domains, prefixes, hosts).' It uses a specific verb and resource, and lists the components dumped, clearly distinguishing it from sibling tools like carrierroute_activate_host which modify state.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool vs alternatives like carrierroute_reload or carrierroute_activate_host. The description only states what it does, not the appropriate context or conditions for its use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, description carries full behavioral burden. Only states 'reload' but does not disclose side effects on active calls, concurrency safety, or whether it's safe to run repeatedly.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, front-loaded sentence conveying the core action efficiently. Could be more informative without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a zero-parameter tool with output schema, but lacks details on return value, side effects, or relation to other call center tools (e.g., cc_agent_login).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so description need not add param info. Baseline 4 is appropriate since schema coverage is 100% and description adds context about what is reloaded.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb (reload) and the resource (call center agents and flows from the database), distinguishing it from sibling reload tools like carrierroute_reload or dialplan_reload.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied (reload after DB changes), but no explicit guidance on when to use versus alternatives or prerequisites (e.g., agent login state).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must fully disclose behavior. It mentions generation and validation steps but does not clarify if the config is saved to disk or just returned, nor does it state side effects (e.g., file creation, modification). Missing details on error handling beyond including raw output.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is structured with a concise summary followed by parameter details. It is front-loaded and to the point, though the parameter descriptions could be more succinct. No wasted sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, the description covers purpose and basic mechanics. However, it lacks clarity on return format (though output schema exists), side effects, and error behavior beyond raw output. Adequate but incomplete for a moderate-complexity tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description carries the burden. The parameter descriptions add value: scenario as template name, params as template parameters, max_attempts as reserved for auto-fix. These meanings are not inferable from schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool generates an OpenSIPS config and iteratively validates it using opensips -C -f. It specifies the result includes validation results and raw output on failure. This distinguishes it from siblings like cfg_generate (which likely generates without validation) and cfg_validate (which only validates).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions iterative validation and includes raw output on failure, but does not explicitly guide when to choose this tool over others like cfg_generate, cfg_validate, or cfg_lint. Given many sibling tools, explicit usage context is lacking.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden but only states the basic action. No disclosure of read-only nature, potential side effects, or other behavioral traits beyond the obvious.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded, zero waste. Perfectly concise for the tool's simplicity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given output schema exists and no parameters, the description is sufficient. However, it could mention the return type explicitly for extra clarity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist; schema coverage is 100% trivially. Baseline score of 4 applies as per rules.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool returns all available lint rule IDs with a specific verb and resource, distinguishing it from sibling tools like cfg_lint or cfg_list_modules.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives. For a simple listing tool, it's somewhat self-explanatory, but explicit usage context is missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description only says 'show', implying read-only, but does not mention any behavioral traits like returned data structure or performance implications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, front-loaded sentence with no wasted words. Perfectly concise for a parameterless tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and an output schema, the description adequately covers the tool's purpose. However, it could hint at the output format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters, so baseline 4 applies. The description does not need to add meaning beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool shows cluster topology (node interconnections). The verb 'show' is specific, and the resource 'cluster topology' is distinct from siblings like cluster_status which focuses on health.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives such as cluster_status or cluster_sync_check. The description lacks context on usage scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and description only mentions triggering an MI command without explaining side effects, safety, or impact on ongoing calls.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, focused sentence with no extraneous information; perfectly concise and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Simple tool with no parameters and an output schema (not described); description adequately covers the core action, though could mention typical use cases or prerequisites.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters in schema, so description cannot add parameter meaning. Baseline of 4 is appropriate for zero-parameter tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action (reload) and resource (dialplan rules from database), distinguishing it from sibling tools like dialplan_add_rule or dialplan_list_rules.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives; for example, it doesn't mention that it should be used after modifying dialplan rules via other tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions 'via MI command' but does not disclose behavioral traits such as whether the reload is destructive, requires authentication, or has side effects. Insufficient for an unannotated tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence of 12 words, front-loaded with the action and resource. No wasted words. Highly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no parameters and presence of an output schema, the description is minimally adequate. However, it lacks details like whether the operation is synchronous, what happens on failure, or logging behavior. For a reload tool, more context would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has no properties (0 parameters), and description coverage is 100%. The description adds no extra param info because none is needed. Baseline score of 5 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Reload the domain table in OpenSIPS via MI command', identifying the specific verb (reload), resource (domain table), and mechanism (MI command). This distinguishes it from sibling reload tools like 'carrierroute_reload' or 'dialplan_reload'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. It does not specify prerequisites, conditions, or exclusions, leaving the agent without context for appropriate invocation among many reload siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; the description carries full burden. 'List all currently banned IPs' indicates a read-only operation. However, no additional behavioral traits are disclosed (e.g., rate limits, auth needs, or that it reflects current state). The presence of an output schema helps but is not part of the description.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with zero redundancy. It is appropriately front-loaded and concise for a simple list tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one optional parameter, straightforward purpose) and the presence of an output schema, the description is minimally adequate. However, it lacks usage guidelines and parameter context, making it incomplete for an agent unfamiliar with fail2ban. Sibling tool count is high, but the tool is distinct enough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It fails to mention the single parameter 'jail' (with default 'opensips'), leaving the agent unaware of how to specify the jail name or that a default exists. The parameter name is obvious, but the description adds no semantic value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses specific verb+resource: 'List all currently banned IPs for a fail2ban jail.' It clearly distinguishes from sibling tools like fail2ban_ban_ip (ban action), fail2ban_unban_ip (unban action), and fail2ban_jail_status (different purpose).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The purpose is clear, but there is no explicit guidance on when to use this tool versus alternatives (e.g., fail2ban_jail_status). The description implies usage when listing banned IPs, but no exclusions or context on scope (e.g., per-jail vs global).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should disclose behavioral traits. It only states the function, missing details like read-only nature, permissions, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no unnecessary words, effectively conveying the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the zero-parameter input and an existing output schema, the description adequately covers the tool's functionality, though it could hint at typical use cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No input parameters exist, so schema coverage is perfect. The description adds context about the output (PID and description), which is helpful.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists all running OpenSIPS processes with PID and description, distinguishing it from other list tools like get_statistics or subscriber_list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives or any prerequisites or exclusions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose any behavioral traits such as side effects, authentication requirements, or potential errors. It only states the generic retrieval purpose, leaving the agent without crucial context for safe invocation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with a short paragraph and parameter list. It is front-loaded with the purpose. Could be improved with a more structured format (e.g., bullet points for parameters) to enhance readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that an output schema exists, the description does not need to detail return values. However, it lacks information on prerequisites, such as whether OpenSIPS must be running. For a simple retrieval tool, it is minimally adequate but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description compensates by explaining the 'group' parameter with examples (e.g., 'core', 'shmem') and the optional 'name' parameter. However, it could be more precise about the format of 'name' and whether it is case-sensitive.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies the verb 'Retrieve' and resource 'OpenSIPS runtime statistics', and provides parameter examples. It is unambiguous and distinguishes itself from more specific sibling tools like 'get_memory_stats'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for general statistics retrieval but does not explicitly state when to use this tool versus alternatives like 'get_memory_stats' or 'get_process_list'. No when-not-to-use guidance is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits such as read-only nature, authorization requirements, or potential side effects. The description carries the full burden but only states the basic function.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no wasted words, effectively conveying the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the description does not need to explain return values. It adequately states that it lists all destination groups and their state. For a simple list tool, this is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so the schema coverage is 100%. The description is sufficient as there are no parameters to document. Baseline score of 4 is appropriate given zero parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'List' and identifies the resource 'load balancer destination groups' and 'their current state', clearly distinguishing it from sibling tools like lb_reload, lb_resize, and lb_set_status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. While the purpose is clear, there is no mention of when to use it instead of other list tools like address_list or dispatcher_list_db.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses permission enforcement and the requirement for command registration, but does not mention side effects, error handling, idempotency, or rate limits. This is insufficient for a tool that executes arbitrary commands.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is fairly concise and well-structured, with a clear purpose and parameter section. No redundant information, but could be slightly tighter without losing clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has an output schema, which reduces the need to describe return values. However, with no annotations and many siblings, the description could be more complete by mentioning typical command output or limitations. It provides adequate but not comprehensive context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains 'command' as the MI command name with examples (ul_dump, lb_list) and 'params' as optional parameters, adding significant meaning beyond the schema. However, it could detail the format of params.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Execute a known OpenSIPS MI command,' using a specific verb and resource. It distinguishes itself from sibling tools like mi_list_commands, which lists commands, and specific command tools like ul_dump, by being the general executor.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains that commands must be registered in MI_COMMANDS and details permission enforcement, but does not explicitly guide when to use this tool versus the many specific sibling MI command tools. It provides context on permissions but lacks alternative recommendations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. It mentions the optional stats snapshot but does not state whether the operation is destructive, requires permissions, or has side effects. This partial disclosure is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with a clear purpose sentence, followed by a structured parameter list. It is concise without being terse, though the parameter list adds length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the main purpose and parameter details. However, given the moderate complexity and absence of annotations, it lacks guidance on usage context and behavioral effects. An output schema exists, so return values are not needed in the description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds significant value beyond the input schema by enumerating valid module values and explaining the verify_stats parameter's purpose and effect. Schema coverage is 0%, so the description compensates well.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool reloads one data table and optionally snapshots stats, using a specific verb and resource. It distinguishes from siblings like reload_data_tables by focusing on a single table and listing supported modules.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
While the parameter documentation lists supported modules, there is no explicit guidance on when to use this tool versus alternatives like reload_data_tables or module-specific reloads. Usage is implied but not contrasted.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should fully disclose behavior. It only says 'enable or disable' without explaining side effects (e.g., impact on active sessions), permission requirements, or confirmation of state change.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with a front-loaded action line followed by parameter details. Minor formatting could improve readability, but it's efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple toggle tool, the description covers the basics but lacks details on error conditions (e.g., invalid URL) or post-state. With an output schema, return values don't need explanation, but more context on behavior would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds significant value beyond the schema by providing an example URL and clarifying the boolean 'enabled' parameter. Schema coverage is 0%, so this description compensates well.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool enables or disables an RTPEngine instance, with a specific verb (enable/disable) and resource. It distinguishes itself from siblings like rtpengine_reload, rtpengine_sessions, etc., which perform different actions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when toggling enablement but does not explicitly state when to use over alternatives like rtpengine_reload or rtpengine_show. No guidance on preconditions or consequences.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. It mentions that the re-registration is 'immediate' and 'forced,' but does not describe side effects, permissions required, error conditions, or what happens to existing registrations. This lack of detail leaves significant ambiguity for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: the first states the primary action, the second provides usage context. Every word is functional, no redundancy, and it is front-loaded with the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has one parameter, no annotations, and an output schema (so return details are covered), the description provides the core purpose and use cases. However, it lacks explanation of the 'aor' parameter, prerequisites, or potential errors, which are needed for full contextual completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has one required parameter 'aor' with 0% description coverage. The description mentions 'AOR' but does not explain its format, purpose, or constraints (e.g., must be a valid URI). Baseline for 0 parameters is 4, but with 1 parameter and no schema coverage, the description fails to add meaningful semantic value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Force an immediate re-registration') and resource ('AOR'), clearly distinguishing it from sibling tools like uac_reg_enable, uac_reg_disable, or uac_reg_list. It directly states what action is performed and on what object.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states two scenarios for using this tool: after credentials change or when an upstream carrier invalidates an existing registration. This provides clear context, though it does not explicitly list when not to use it or mention alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full behavioral burden. It states it 'dumps' the table, implying a read operation, but does not disclose potential performance impacts, resource usage, or safety characteristics. The brief parameter behavior is noted, but broader behavioral traits are missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: one sentence for purpose and a bullet for the parameter. No extraneous information. Front-loaded with the primary action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple dump tool with one optional parameter and an output schema present, the description is largely complete. It explains the main intent and parameter effect. Could mention system impact or distinguish from similar tools, but overall acceptable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds value by explaining the 'brief' parameter: 'If True, return a compact summary instead of full details.' The schema only provides type and default, so this explanation is necessary. Schema description coverage is 0%, so the description compensates well.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('dump') and resource ('entire user location table'), clearly distinguishing from sibling tools like 'ul_show_contacts' or 'mid_registrar_dump' that operate on more specific subsets.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as 'ul_show_contacts' or 'mid_registrar_dump'. The description does not provide context for choosing this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose any behavioral traits beyond the basic purpose. It lacks information on side effects, permissions, rate limits, or any constraints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise with two short sentences and a parameter list. Every part is essential; no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple query tool with an output schema (not shown but indicated), the description adequately covers purpose and parameters. Some context about the return format is missing but likely covered by the output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 0%, but the description adds meaningful explanations for both parameters (aor and table), including an example for aor and default value for table. This compensates for the schema gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool 'Show registered contacts for an Address of Record (AOR)', with a specific verb and resource. It distinguishes from sibling tools like ul_add_contact or ul_remove_contact.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives, nor any mention of when not to use it. The description only lists parameters without contextual usage advice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Given no annotations, the description adds valuable behavioral traits: the operation applies at runtime and does not require a reload. It does not disclose error handling or what happens if the host is missing, but the runtime note is helpful.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first states purpose with example, second adds runtime behavior. No unnecessary words, front-loaded key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 5 required parameters and no schema descriptions, the description lacks essential details for correct invocation. The output schema exists but is not leveraged. The agent cannot confidently infer parameter semantics from the description alone.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and the description does not describe any of the 5 parameters. The parameter names (carrier, domain, prefix, host, new_host) provide some clues, but no explanation of expected formats or relationships is given.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Replace a host in carrier route' with an example 'IP change'. It uses a specific verb-resource combination and distinguishes from sibling tools like carrierroute_activate_host or carrierroute_reload.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions 'Does not require a reload; applies at runtime', implying use when a runtime change is needed without reload. However, it does not explicitly state when to use this tool versus alternatives, nor when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It implies a read-only operation (computing a diff) but does not explicitly state that no modifications occur or detail any side effects. The nature of diffing is generally safe, but more explicit transparency would improve the score.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: one sentence for purpose and a brief parameter list. Every word contributes value with no redundancy. The structure is front-loaded with the core function and immediately followed by parameter details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the low complexity (two string parameters) and existence of an output schema, the description is marginally adequate. However, it does not explain the output format (e.g., returns a string or file path) or provide examples. The output schema may cover this, but the description alone lacks completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 0%, so the description carries the full burden. It provides clear explanations for both parameters (config_a as original, config_b as modified) beyond the schema's bare names. This adds meaningful semantic context, though it lacks format examples or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool computes a unified diff between two OpenSIPS configurations. It uses a specific verb ('Compute') and resource ('unified diff between two OpenSIPS configurations'), and distinguishes itself from sibling tools like cfg_diff_reference by being the basic diff operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool compared to other cfg_* siblings, such as cfg_diff_reference, cfg_lint, or cfg_parse. There is no mention of use cases, prerequisites, or alternatives, leaving the agent without context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; the description carries the full burden. It indicates the tool returns structured data from parsing, implying a read-only operation, but does not explicitly state side effects, permissions needed, or rate limits. The behavioral disclosure is minimal but not misleading.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short and well-structured: a headline sentence, a bullet-like list of return types, and a clear parameter explanation. Every part adds value. Could be slightly more concise by removing the redundant 'Returns' line, but overall efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (not shown), the description does not need to detail return formats. It covers the tool's basic function and parameter. For a simple parse tool with one parameter, this is adequate, though it could mention if the parsing is syntax-only or semantic.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description compensates by clarifying that 'config_content' expects the full OpenSIPS configuration text. This adds meaning beyond the parameter name alone, though it does not describe format or constraints beyond 'full'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool parses an OpenSIPS configuration and extracts its structure, listing specific output categories (modules, modparams, listen addresses, routes, global params). This distinguishes it from sibling cfg_* tools like cfg_edit or cfg_lint, which modify or validate instead of parsing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. While the purpose is clear, the description omits context such as prerequisites, when to prefer cfg_parse over cfg_explain or other parsing tools, or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must cover behavior. It states listing and filtering, but does not disclose any specific behaviors like rate limits, authentication, or side effects. For a read-only list, it's acceptable but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with the action, no redundant information. Every word contributes to the purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has an output schema (not shown) which likely covers return format, but the description does not mention pagination via limit/offset. For a simple list tool, it is adequate but could be more complete by noting pagination behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Description explains 'filtered by group or prefix' which adds meaning to groupid and prefix parameters, but does not mention limit and offset. With 0% schema description coverage, the description partially compensates for two of four parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (List), the resource (dynamic routing rules), and the optional filters (by group or prefix). It distinguishes from sibling tools like drouting_add_rule or drouting_delete_rule.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage via 'optionally filtered by group or prefix', but does not provide when to use this tool vs alternatives like drouting_list_carriers or drouting_list_gateways. No exclusions or when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits such as side effects, authentication requirements, or return format. For a status-checking tool, it should at least indicate it is read-only.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence is concise and front-loaded with the key action. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool has no parameters and an output schema (not shown but present). Description is sufficient for a simple status check; no additional context seems needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has no parameters, so the description does not need to add parameter details. Baseline for 0 parameters is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description uses a specific verb 'Check' and identifies the resource 'OpenSIPS Control Panel availability and status,' making it clear what the tool does. It is distinct from sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., health_check, get_uptime). No mention of prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description bears full burden. 'Force cleanup' suggests destructive action (deletion) but doesn't disclose reversibility, side effects on active subscriptions, or idempotency. Minimal beyond the literal action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with verb and resource, no fluff. Every word adds meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and output schema exists, description is minimally adequate but lacks behavioral context for a cleanup operation. Could benefit from notes on impact or when to use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist (0 params, 100% schema coverage). The description doesn't add parameter info but is not required to. Baseline 4 is appropriate for a param-less tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb 'Force cleanup' and resource 'expired presence records (watchers + presentities)' clearly identify the action and scope, distinguishing it from sibling tools like presence_list_subscriptions or presence_refresh_watchers.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like presence_refresh_watchers. It implies usage for expired record cleanup but lacks explicit context or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden. It states 'Perform a basic security audit' which implies a read-only analysis, but does not explicitly confirm that no changes are made, nor does it mention any authorization requirements or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured with a clear sentence, followed by a Parameters section and Returns section. All information is relevant and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (basic security audit on a config), the description covers the main inputs and outputs, including the return structure with 'findings' and 'summary counts'. No output schema was provided in the input, but the description mentions it. However, it lacks details on what specific issues are checked.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter 'config_content' has a clear description: 'The OpenSIPS configuration text to analyse.' This adds meaning beyond the schema's simple type declaration, compensating for the 0% schema description coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Perform a basic security audit on an OpenSIPS configuration snippet.' This provides a specific verb and resource, and distinguishes it from sibling tools like cfg_lint or security_generate_ha1.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. There is no mention of when not to use it or which other tools might be better suited for different types of security checks.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must fully disclose behavior. It only states 'bridge' but does not explain side effects, reversibility, or what happens to the original call. This lack of detail is critical for a mutation operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with a clear summary, use cases, and a structured parameter list. It is well-organized but the parameter section could be more integrated into the narrative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is adequate for a simple two-parameter tool, mentioning the source for the key and target format. However, it does not address error conditions or what the output schema returns, though the latter is partially covered by the existence of an output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaningful context beyond the schema: it explains that 'key' comes from b2b_list_sessions and provides an example format for 'new_dst' (sip:agent@domain). This compensates for the 0% schema description coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Bridge an existing B2B call to a new destination URI' with specific use cases (attended-transfer, failover, music-on-hold). It uniquely identifies the tool among siblings like b2b_terminate and b2b_trigger_scenario.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description lists explicit use scenarios and references b2b_list_sessions for obtaining the session key. However, it does not explicitly state when not to use this tool or mention alternative tools, but the context is clear enough for typical usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description implies read-only list operation but does not explicitly state no side effects or other behavioral traits. With no annotations, description carries full burden; minimal but acceptable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence efficiently conveys purpose and scope. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Sufficient for a simple list operation with no parameters and existing output schema. Lacks clarification on 'active' definition but overall adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters in input schema (coverage 100%). Description adds no param info, but baseline is 4 for zero-parameter tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'List active B2B logic sessions' and specifies included attributes (entity keys, state, legs). Distinguishes from sibling tools like b2b_list_entities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives (e.g., b2b_list_entities, b2b_terminate). Description does not provide context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description implies a read-only, non-destructive action, but no explicit behavioral traits are disclosed beyond that. With no annotations, the description is minimally adequate for a simple list operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no redundant information. It is concise and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and likely returns a list of subscriptions (output schema exists), the description is brief but covers the essential functionality. It is complete for a straightforward list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameters, so the description adds no parameter-specific information. According to the baseline for 0 parameters, this is sufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies the verb 'List' and the resource 'active Shared Call Appearance (SCA) subscriptions'. It is specific and distinguishes this tool from siblings like b2b_bridge or b2b_list_entities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It does not mention related operations or conditions that would warrant its use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must convey behavioral traits. It states the tool activates and re-enables, which is accurate, but does not disclose any side effects, permissions, or state checks. It provides minimal but correct behavioral info.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences, front-loaded with the action, and contains no unnecessary words. It is highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 4 required parameters and no behavioral annotations, the description provides only the basic purpose and usage scenario. It lacks information on prerequisites, errors, or further details, making it adequate but not thorough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning the description adds no information about parameters. The parameter names (carrier, domain, prefix, host) are somewhat self-explanatory, but the description fails to compensate for the lack of schema documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Activate' and the resource 'carrier route host', and uses the phrase 'return to service' which distinguishes it from its sibling 'carrierroute_deactivate_host'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Use this after maintenance to re-enable a previously deactivated gateway', giving a clear context of use. However, it does not mention when not to use it or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description carries the burden. It discloses the insertion location (after last loadmodule/modparam block) but lacks details on validation, error handling, or side effects. Could be more transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a brief intro and parameter list. It is clear but could be slightly more concise (e.g., avoid repeating 'loadmodule' twice).
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With many sibling tools and a complex operation, the description covers the core purpose but does not mention return values or post-conditions. The presence of an output schema mitigates this partially, but additional context on expected results would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 0% schema coverage, the description details each parameter: config_content as existing config text, module_name as the module string, and params as an optional dict. This adds significant meaning beyond the empty schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it adds a module with optional parameters to an OpenSIPS configuration, specifying the insertion of loadmodule and modparam directives. It distinguishes itself from sibling tools like cfg_edit or cfg_lint.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool over alternatives (e.g., when to use cfg_add_module vs module_add_param or cfg_edit). The usage is implied but not stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It mentions that validation runs 'opensips -C' when available, but it does not describe side effects, whether files are modified, permissions needed, or error handling. The description lacks sufficient behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a concise introductory paragraph and a clear parameter list. It is front-loaded with the core purpose. However, some phrases are slightly verbose (e.g., 'Materialises the Kamailio-style flag vocabulary') and could be more direct.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the description does not need to detail return values. It adequately explains parameters and validation. However, it lacks error cases, prerequisites, and permission requirements, which would improve completeness for a tool with no annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, but the description compensates by explaining each parameter: flags are any combination of WITH_* flags, site_params lists specific keys like LISTEN and DB_URL, and validate runs opensips -C. This adds substantial meaning beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Build a flat opensips.cfg from a set of WITH_* flags.' It further explains the benefit over nested ifdefs, distinguishing it from sibling tools like cfg_compose_flags or cfg_generate. The verb 'build' and resource 'opensips.cfg' are specific.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when you have WITH_* flags and want a clean config, but it does not explicitly state when to use this tool over alternatives like cfg_compose_flags or cfg_generate. It references cfg_list_flags for available flags but lacks guidance on when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behaviors like read-only nature, permissions, and side effects. It only describes the listing operation and parameters, omitting whether it requires file access or has any restrictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is brief and well-structured, with a summary sentence followed by parameter descriptions. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given an output schema exists, return values need not be described. The description explains the main functionality and parameters, but could be more explicit about search scope (e.g., non-recursive). It is adequate for a list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description adds significant meaning: config_path is 'Absolute path whose backups we want to enumerate' with basename as prefix filter, and backup_dir is optional defaulting to config_path's directory.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists .bak-* backups for a config path, with a specific verb and resource. It distinguishes from siblings like cfg_backup (create) and db_table_list_backups (database).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implicitly indicates usage for enumerating config backups, but lacks explicit guidance on when to use this vs alternatives like cfg_backup or db_table_list_backups.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full responsibility. It notes Kamailio compatibility, which is helpful, but does not disclose if the list is read-only, the output format, or any side effects. Minimal transparency for a simple list operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loading the core purpose with no unnecessary words. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and an available output schema, the description is reasonably complete. It could mention the output is a list of flag names, but the schema covers that. Slight gap in usage context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, earning a baseline of 4. The description adds no parameter information, which is acceptable as none exist.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists all available 'WITH_*' feature flags, using a specific verb and resource. This distinguishes it from sibling cfg_list_* tools that list modules, scenarios, or versions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidelines on when to use this tool versus alternatives are provided. The description does not mention context, prerequisites, or when to avoid using it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It implies a read-only operation by using 'List', but does not explicitly state that it is non-destructive or safe. It also does not disclose any authentication requirements or performance impact.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that conveys the essential information without any fluff. Every word is necessary and contributes to clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no input parameters and an output schema exists, the description is largely sufficient. It states that all scenarios are listed with descriptions. However, it could be slightly more complete by explicitly stating it is a read-only operation or clarifying the scope (e.g., 'all' is already present).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters with 100% schema description coverage, so the description does not need to add parameter meaning. Baseline for zero parameters is 4, and the description does not attempt to add unnecessary param info.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List', the resource 'available OpenSIPS configuration scenarios', and mentions they come with descriptions. It effectively distinguishes from sibling tools like docker_list_scenarios or sipp_list_scenarios by specifying 'OpenSIPS configuration scenarios'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as docker_list_scenarios or sipp_list_scenarios. It does not mention any prerequisites, limitations, or context for usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description must disclose behavior. It states this is a diagnostic check (read-only) and mentions return fields, but doesn't elaborate on side effects, permissions, or error conditions. Adequate but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with 'Diagnostic', no extraneous information. Every sentence serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple parameterless check tool with an existing output schema, the description adequately covers purpose and outputs. Could mention typical use case (e.g., before generating m4 configs) but not necessary.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so schema coverage is 100%. The description adds value by specifying what the command returns (path, version, status), providing meaning beyond the empty schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'check' and identifies the exact resource 'm4 binary availability'. It distinguishes this tool from sibling cfg_* tools by targeting a specific binary dependency, not general configuration manipulation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, such as cfg_check_compat or other diagnostic tools. Contextual cues are missing, leaving the agent to infer applicability.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It describes a read-only list operation, but does not explicitly state it is non-destructive or if it requires any permissions. Adequate but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded, no extraneous words. Efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Simple tool with no parameters and a clear purpose. Description is sufficient for the complexity level, though it could mention it is read-only or that it returns live data. With no output schema details, it's reasonably complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters in schema, so schema coverage is 100%. Description adds value by clarifying the output: lists nodes and their status. Baseline for zero parameters is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it lists all cluster nodes and their status. Differentiates from sibling tools like cluster_topology or cluster_sync_check by focusing on current status of all nodes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus siblings such as cluster_drain_node or cluster_promote_node. Implicitly for viewing status, but no exclusions or alternatives mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, description partially informs that operation is read-only by stating 'Return the docker-compose command', but lacks details on prerequisites, error handling, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, no extraneous information. Highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given presence of output schema (inferred from context), description is adequate. Could be improved by explicitly stating that the command is not executed and by suggesting docker_list_scenarios for scenario discovery.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Description adds examples of valid scenario names (e.g., load-balancer, class4-sbc), which provides context beyond the schema's minimal title. However, does not reference the sibling tool docker_list_scenarios for available names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'Return' and resource 'docker-compose command to deploy a specific scenario'. Differentiates from sibling tools like docker_list_scenarios and sipp_run_scenario.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. Does not specify that it only returns the command without executing deployment, nor contrasts with similar tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description fully relies on itself. It states it lists scenarios and returns names and paths, implying a read-only operation. However, it does not explicitly confirm no side effects or disclose any behavioral details beyond that.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no extraneous information. The first sentence states the purpose, the second describes the output. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and an output schema exists, the description adequately covers what the tool does and returns. It could provide more context about where scenarios come from or if any environment setup is needed, but it is sufficient for a simple listing tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so the description adds value by explaining the return structure (names and file paths). Baseline for zero parameters is 4, and the description meets this.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists all available Docker deployment scenarios and specifies what is returned (scenario names and docker-compose file paths). The verb 'list' and resource 'Docker deployment scenarios' are specific and distinguishable from sibling tools like docker_deploy_scenario.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, such as before deploying a scenario or to check available scenarios. There is no mention of prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It implies a read-only operation via 'list' but does not explicitly state it is non-destructive or mention any side effects. The minimal disclosure is adequate but not enriched.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: one line for purpose plus a parameter section. There is no filler, and the key information is front-loaded. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool, the description provides enough context: it explains filtering parameters and what the tool does. The output schema exists to document return values. A minor gap is no definition of 'active', but overall it is complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides no parameter descriptions (0% coverage), so the description adds essential meaning: examples for 'event' (e.g., E_DLG_LOAD) and 'socket' (e.g., udp:1.2.3.4:8080). This clarifies format and usage beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists active event subscribers with optional filtering. The verb 'list' and resource 'event subscribers' are specific, and it distinguishes from sibling tools like 'event_list' (lists events) and 'event_subscribe' (subscribes to events).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. Although it mentions optional filtering, it does not explain when to choose this over related tools like 'event_subscribe' or 'event_list', lacking exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. 'Get status' correctly indicates a read operation with no side effects. However, no further behavioral info (e.g., permissions, dependencies) is given. Adequate but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with immediate purpose, no filler. Front-loaded verb and target. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Output schema exists (likely defines return fields), so description needn't detail output. One optional parameter with default covers common usage. Missing error scenarios or prerequisites, but for a simple status tool, this is mostly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% (no description in schema), so description must compensate. It adds examples ('e.g., opensips, opensips-auth') which give context for the 'jail' parameter, but doesn't explain its purpose or accepted values. Baseline 3 due to low coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Get status of a specific fail2ban jail' with verb 'Get' and resource 'status of a specific fail2ban jail'. Distinguishes from the sibling fail2ban_status by targeting individual jails, and provides concrete examples (opensips, opensips-auth).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies usage for specific jail status but lacks explicit guidance on when to use this versus sibling tools like fail2ban_status (for overall status) or fail2ban_ban_ip. No when-not-to-use or alternative references.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the behavioral burden. The verb 'dump' implies a read-only operation, but the description does not explicitly state that it is non-destructive or safe to run. Minimal adequacy.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words. It is front-loaded with the verb and resource, making it immediately clear what the tool does.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has zero parameters and the presence of an output schema, the description is nearly complete. It could optionally mention that the dump is from memory (already implied) but is sufficient for the use case.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so the baseline is 4. The description does not need to add parameter information since the schema already covers it completely.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly uses the verb 'Dump' with a specific resource ('LCR rules and gateways currently loaded in memory'), distinguishing it from siblings like lcr_reload which loads from database.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives like lcr_reload. The agent must infer from context that dump is for viewing current state, but no direct comparison is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description must disclose behavior. It correctly indicates a read-only list operation with no side effects. However, it does not mention the output format or any performance implications, though the tool is simple.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, well-formed sentence that immediately conveys the tool's purpose. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless list command with an output schema, the description is sufficient. It tells the agent exactly what the tool does. However, it could mention that it returns a list of command names for reference.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters with 100% coverage (since none exist). Baseline for zero parameters is 4. The description correctly implies no parameters by stating 'list all' without any qualifiers.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'all MI commands', distinguishing it from sibling list commands (e.g., address_list, domain_list) which list specific entities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. The description does not specify that it is a discovery tool for available commands, nor does it mention that it has no parameters.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility for behavioral disclosure. It only states the tool builds a bundle from cfg, but does not mention side effects, permissions, rate limits, or any destructive potential. This is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two sentences, no filler, and the key action is front-loaded. Every sentence is meaningful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has six parameters, no annotations, and an output schema exists, the description adequately covers purpose and usage but lacks behavioral and parameter detail. It references the output shape via another tool, which helps, but overall is minimally complete with clear gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It only mentions that the tool takes 'the cfg directly' but does not explain any of the six parameters, their meanings, or constraints. Parameter names are somewhat self-explanatory but the description adds no value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool builds a full deployment bundle from an arbitrary cfg, not a scenario. It also mentions the output shape matches another tool, distinguishing it from the sibling observability_generate_bundle.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use it: after hand-editing generator output or migrating legacy configs, implying not to use it with scenarios. This provides clear usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description implies a read-only operation ('Retrieve'), but does not explicitly state safety, permissions, or data freshness. Adequate but not extra.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences front-loading the purpose, with specific details in the second. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With zero parameters and an output schema, the description sufficiently outlines what statistics are returned. Could mention it returns live data, but overall complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters (baseline 4). Description adds no param info, but none needed due to empty schema. Schema coverage is 100%.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description specifies 'Retrieve registrar-related statistics' and details 'registration counts, expiry stats, and related counters' from specific modules, clearly distinguishing it from other stats tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus other stats tools (e.g., sl_stats, nathelper_stats) or alternative methods (e.g., ul_dump). Missing context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description should disclose behavioral traits. It only says 'Run a SIPp load test' without detailing side effects, resource requirements, or return behavior. Minimal transparency beyond the function name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: a single introductory sentence followed by a clean parameter list. Every sentence is informative and non-redundant, fitting in a small space.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (not shown), the description adequately covers parameters and overall purpose. It could mention what output to expect, but with the output schema, this is not a critical omission.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description provides meaningful explanations for all four parameters (target, cps, duration, scenario), including default values and allowed formats. This compensates well for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Run a SIPp load test', specifying the verb and resource. It distinguishes from sibling tools like sipp_list_scenarios and sipp_run_scenario by focusing on load testing with custom parameters.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for load testing but provides no explicit guidance on when to use this tool versus alternatives like sipp_run_scenario. No 'when not to use' or prerequisite information is included.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must disclose behavioral traits. It only states the action and parameters, not side effects, permissions, or rate limits. Lacks detail for a run tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is a single sentence followed by a concise parameter list. Every sentence is purposeful, no fluff, and front-loaded with the main action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (not examined), the description adequately covers execution. Could mention prerequisites like file validity, but not essential.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description fully explains each parameter (scenario_file, target, options) with examples, adding significant value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Run a SIPp XML scenario file against a target SIP server' clearly states the action (run), resource (SIPp XML scenario), and target, distinguishing it from siblings like 'sipp_list_scenarios' and 'sip_generate_sipp_scenario'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives (e.g., 'sipp_load_test'). Only implied usage from the tool's name and description, with no exclusions or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses the capture behavior (network interface, duration with 300s max, BPF filter, output path restriction to /tmp or /var/tmp). However, it does not clarify whether the tool blocks until capture completes, overwrites existing files, or requires cleanup. The behavioral transparency is adequate but incomplete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: a one-line summary followed by a clear table of parameters with defaults and constraints. Every sentence provides necessary information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 4 parameters, no annotations, but has an output schema, the description is fairly complete. It explains the capture purpose, parameter defaults and constraints, and output path rules. It lacks details on return behavior (but output schema exists) and does not mention if capture is synchronous or has side effects, but overall it is sufficient for basic usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It provides detailed parameter descriptions including defaults, constraints (duration max 300, output path must resolve under /tmp/...), and the filter expression default. This adds significant meaning beyond the schema, which only contains defaults and types. A higher score would require even more context like format expectations or side effects.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool captures SIP traffic using tcpdump with sngrep-compatible pcap output. The verb 'Capture' and resource 'SIP traffic' are specific, and the tool is distinguished from siblings like homer_export_pcap or trace_start_live by focusing on live capture via tcpdump.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives or when not to use it. There is no mention of prerequisites, typical use cases, or exclusions. The description only states what it does, not the context of use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden. It implies a read-only list operation but does not explicitly state behavioral traits like safety, ordering, or performance. However, the tool is simple and the description is minimally adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence with no wasted words. Every word is necessary and informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and an existing output schema, the description is mostly complete. It could mention the scope more explicitly, but it is sufficient for a list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters in the input schema and schema description coverage is 100%. The description has no need to add parameter info; baseline 4 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'List' and resource 'TLS domain configurations' with scope 'all from the database'. It clearly distinguishes from sibling tools like tls_add, tls_delete, and tls_update which perform different operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. Among many TLS-related siblings, there is no differentiation of context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries the full burden. It explains that the tool disables and stops re-registration, but does not disclose side effects, permissions, or error conditions. Some behavioral context is provided, but it is not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no unnecessary words. It is front-loaded with the main action. However, it could benefit from additional structure, such as separating the action from the clarification.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple (one parameter), and the description provides the core purpose. However, it does not mention return values or status, and given the existence of an output schema, some information is missing. It is minimally complete for its complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has no descriptions for the 'aor' parameter (coverage 0%). The description indicates that 'aor' is the address of record, adding meaning beyond the schema. However, it does not specify format or constraints, so it is partially helpful.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Disable'), the resource ('UAC registration'), and the key parameter ('by AOR'). It also explains the consequence ('stops re-registering'), distinguishing it from sibling tools like uac_reg_enable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for stopping re-registration but does not explicitly state when to use this tool versus alternatives like uac_reg_enable or uac_reg_force_register. No exclusions or prerequisites are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description states it shows current carrier/trunk registrations, implying read-only behavior, but does not explicitly confirm non-destructiveness, permissions needs, or rate limits. The output columns are listed, adding some transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with purpose and optional filters, followed by clarification of the data shown and output columns. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with two parameters and an available output schema, the description covers the purpose, filters, and output fields. It does not explicitly state default behavior (list all when no filters), but that is reasonable. Minimal gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning to the schema by clarifying that the two optional parameters are filters for AOR and contact, which the schema alone (with only names and types) does not provide. However, it does not explain the format or semantics of AOR, leaving some ambiguity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists UAC registrations with optional filters by AOR or contact. It specifies the resource (UAC registrations) and verb (list), and distinguishes from sibling tools like uac_reg_enable or uac_reg_force_register by being a read-only list operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions optional filters but provides no guidance on when to use this tool versus alternatives, no exclusions, and no context about prerequisites or complementary tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description implies a read-only operation, but does not explicitly state no side effects, data volume, or other behavioral traits. Minimal disclosure beyond the action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One short, front-loaded sentence with no wasted words. Every word is meaningful and carries the purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and an output schema (not required in description), the description is fully adequate for a straightforward list tool. No additional context needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Zero parameters; schema coverage is 100% trivially. The description adds no parameter information because none exist. Baseline 4 for zero parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'all call center agents' with their 'current state'. This distinguishes it from sibling tools like cc_agent_login (login) and cc_list_queue (queues).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives. No exclusions or prerequisites mentioned. The description only states what it does, not when or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states the tool 'compares' and 'shows which parts are custom,' implying read-only behavior, but does not explicitly state that it does not modify anything. It also does not mention authorization needs, rate limits, or any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: a concise one-sentence summary followed by an explanation and a parameter list. It is front-loaded with the main action. The parameter list is detailed but not overly verbose. It earns a 4, slightly off from perfect because the parameter descriptions could be considered duplicated from what might be in the schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (comparison with output schema) and sibling tools, the description is fairly complete. It explains the output shows 'custom' parts, hints at the use case, and references cfg_list_scenarios for scenario names. However, it does not mention prerequisites like needing a reference template or whether it outputs a standard diff format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It provides useful descriptions for each parameter: 'config_content: The OpenSIPS configuration text', 'scenario: The reference scenario name (see ``cfg_list_scenarios``)', and 'params: Parameters used to render the reference.' This adds significant meaning beyond the schema's titles alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Compare a config against the reference output for a scenario' with a specific verb ('Compare') and resource ('config against reference output'). It distinguishes itself from sibling tools like cfg_diff (which compares two configs) by specifying the comparison target is the reference scenario template.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions it is 'useful for reviewing what your team has diverged on and would need to re-apply after upgrading the reference template,' which implies a use case but does not explicitly state when to use this tool versus alternatives like cfg_diff or cfg_lint. There is no when-not guidance or mention of other tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It describes the checks performed but does not explicitly state that the tool is read-only or non-destructive. The linting nature implies no side effects, but this is not confirmed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear purpose sentence, a bullet list of checks, and a parameter section. Every sentence is informative; no fluff or redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the tool's purpose and parameters adequately. Since an output schema exists, it is not required to describe return values. However, it lacks explicit mention of read-only behavior, which would make it fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Since schema description coverage is 0%, the description compensates by clearly explaining each parameter: config_content is 'The OpenSIPS configuration text to lint', include and exclude are optional rule ID lists with examples. This adds essential meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Run the OpenSIPS semantic linter against a configuration' with a specific verb and resource. It then lists specific checks (module dependencies, orphaned modparams, etc.), distinguishing it from sibling tools like cfg_validate which likely focus on syntax.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives such as cfg_validate or cfg_explain. The description does not mention prerequisites, when not to use, or refer to sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It explains that inputs are file contents (not file paths) and that validation is conditional on opensips binary presence. However, it does not state whether the action is destructive, what the return value is (though output schema exists), or prerequisites like m4 installation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, with a one-line summary followed by structured parameter definitions. No redundant content; every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple 3-parameter tool, the description covers the main action, inputs, and optional validation. Output schema exists so return format is documented. Missing minor context like prerequisites (m4 installation) but overall sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage, but the description fully explains each parameter: main_m4 is 'Contents of opensips.cfg.m4', local_m4 is 'Contents of local.cfg.m4 (site-specific define(...) lines)', validate has detailed conditional logic. This adds critical meaning beyond the schema titles and types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action: 'Run ``m4 local.cfg.m4 opensips.cfg.m4``' and optionally validate. It specifies verb (run) and resource (m4 preprocessing). Among sibling cfg_* tools, this is distinct as it handles m4 preprocessing, not parsing or validation alone.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternative cfg_* tools like cfg_parse or cfg_validate. The description only defines the tool's behavior but does not provide decision criteria or exclusion scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes the output and purpose, but the mention of 'Ask the user five questions' may be misleading since parameters are provided; lacks details on side effects or interactivity.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is relatively long with a separate Parameters section that repeats defaults; could be more concise despite front-loading the purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers overview and output format, but missing error handling or validation details; output schema exists so return values are covered, but overall completeness is adequate but not thorough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema coverage, the description adds meaningful explanations for each parameter (e.g., role options, TLS significance), compensating well for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it generates a PGTune-style capacity-tuned override snippet for OpenSIPS config, distinguishing it clearly from sibling cfg_* tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explains the snippet is meant to be merged into an existing config and is NOT a full configuration, but does not explicitly list when not to use it or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes the behavioral effect: 'flips status back to OK so peers resume sending traffic/replication.' With no annotations, this is adequate but lacks details on permissions, idempotency, or what happens if node is already active.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise, front-loaded sentences. No filler or redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 required params, counterpart named), the description covers the essential purpose and effect. An output schema exists to cover return values, so this is reasonably complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and the description adds no additional meaning to cluster_id or node_id beyond their names. The tool's purpose is clear but parameter guidance is missing.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action: 'Bring a disabled cluster node back into active service.' Names the counterpart cluster_drain_node, distinguishing it from related tools like cluster_status, cluster_sync_check, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly identifies as the counterpart to cluster_drain_node, implying it should be used to reverse a drain. Does not explicitly state when not to use it or provide alternatives, but the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It mentions 'newest first' ordering, which is a behavioral trait. However, it doesn't clarify if the operation is read-only or requires permissions. Given no contradictions and modest disclosure, score is moderate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence of 8 words, front-loaded with purpose. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool has output schema, simple params, no enums or nested objects. Description covers purpose, parameters, and ordering. No gaps given context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description must add meaning. It explicitly links parameters: 'for *table* under *out_dir*', clarifying their roles. This goes beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list' and the resource 'backup files for *table* under *out_dir*', with ordering 'newest first'. It distinguishes from siblings like cfg_list_backups (config backups) and db_table_backup/restore (create/restore).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives like cfg_list_backups or db_table_backup. The description is purely descriptive with no contextual hints.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries full burden. 'List' implies read-only, but the description doesn't explicitly state safety aspects (e.g., no modifications, permission requirements). It adds minimal behavioral context beyond the obvious.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with a single sentence and parameter details, front-loaded with the core purpose. Every part earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple listing operation and the existence of an output schema, the description is mostly complete. It could mention the return format (list of dialog objects), but the output schema likely covers that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds valuable meaning to the parameters (callid for filtering by specific dialog, from_tag for further narrowing) beyond the empty schema descriptions, which have 0% coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List active SIP dialogs' uses a specific verb and resource, clearly distinguishing from sibling tools like dlg_end (which ends dialogs) and dlg_get_profiles (which retrieves profiles).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool vs alternatives, such as when to use this versus other dialog-related tools. No prerequisites or exclusions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description states the behavior: checking health and returning a summary of reachability and responsiveness. However, with no annotations, it lacks details on side effects (assumed read-only), safety, or performance impact. It adds moderate value beyond the tool name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with verb and resources, second sentence clarifies output. No unnecessary words; efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool has no parameters and an output schema, so description is complete enough. It identifies components and output nature, and the output schema presumably details the return format. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters in input schema, so description cannot add parameter semantics. Baseline of 4 is appropriate given no parameters exist and description does not need to explain them.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Check health' and specifies the resource 'all ecosystem components' with explicit list (OpenSIPS, Homer, RTPEngine, CP). This distinguishes it from sibling tools like 'ecosystem_overview' or 'health_check' by being component-specific.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this vs alternatives. For example, there is no mention of when to use this over 'ecosystem_overview' or 'health_check', nor any exclusions or context hints.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must cover behavior. It explains the validation process but does not disclose potential side effects, permissions required, or whether it is read-only. The description is adequate but lacks full transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a brief summary and a clear parameters list. It is concise but could be slightly more compact; no wasted sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity and existence of an output schema, the description covers the core purpose, inputs, and process adequately. It does not explain return values, but the output schema fills that gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although schema description coverage is 0%, the description adds a Parameters section that explains each parameter's meaning (e.g., module_name omits .so extension, module_path specifies directory). This adds value beyond the schema's type and title.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool validates an OpenSIPS module can be loaded, using a specific process (creates minimal config, runs opensips -C). It distinguishes from sibling tools like module_compile or module_scaffold.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage context is implied but not explicitly stated. The description does not provide when-to-use or when-not-to-use guidance compared to alternatives, leaving the agent to infer.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description effectively carries the burden by explaining the consequences of memory exhaustion (silent call kills, leak detection). It adds behavioral context beyond a simple 'report' statement.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: one sentence for purpose, followed by clear explanations of each memory pool. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and an output schema, the description provides sufficient context about the memory pools and their operational significance. It does not describe the output format, but the output schema covers that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters (0), so baseline is 4. The description adds value by explaining the meaning of shm and pkg, which complements the input schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool reports shared-memory and per-process package-memory usage, using specific verbs and resources. However, it does not distinguish from sibling tools like get_memory_stats or perf_hotspots.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage scenarios (shm for production monitoring, pkg for leak detection) but lacks explicit guidance on when to use this tool versus alternatives or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description partially discloses behavioral traits like directory allowlisting and the optional label. However, it does not explicitly state whether the operation is read-only or side-effect-free, nor does it mention performance or permission implications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise at about 100 words, front-loaded with a clear purpose, and structured with a 'Parameters' heading and bullet points. Each sentence adds value without repetition or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 3 parameters, no annotations, and an output schema, the description covers parameter semantics thoroughly and mentions constraint (allowlisted dirs) and filename behavior. Minor gap: it doesn't explicitly state that the snapshot is saved as a JSON file to the directory (though implied), but the presence of an output schema mitigates the need for return value description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description carries full responsibility. It explains all three parameters in detail, including constraints (allowlisted directories), usage hints (label tags), and examples for extra_stat_groups. This adds significant meaning beyond the schema types and defaults.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Capture a structured state snapshot of the running OpenSIPS,' which specifies the verb (capture), resource (structured state snapshot), and context (running OpenSIPS). It is distinct from siblings like snapshot_diff and snapshot_list, which handle existing snapshots.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as snapshot_diff or snapshot_list. The description focuses solely on parameters without mention of prerequisites, exclusions, or when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. It mentions the results are ordered newest first, but does not mention side effects, access requirements, or error conditions (e.g., if the directory doesn't exist). The output schema exists, which explains return values, so some behavioral detail is deferred.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences that state the function and a usage scenario. No unnecessary words or details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (one parameter, has output schema), the description covers the essential purpose and a realistic use case. It does not explain what constitutes a snapshot or the output format, but the output schema likely handles that. Missing some edge case guidance but adequate for most scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter 'out_dir' has no description in the schema (0% coverage). The description only says 'in a directory' without elaborating on the parameter's meaning, format, or constraints. This fails to compensate for the schema's lack of documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'List snapshots in a directory, newest first.' This is a specific verb (list) and resource (snapshots) with ordering and scope. It distinguishes itself from siblings like snapshot_capture or snapshot_diff by focusing on listing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides a concrete use case: 'Useful when you've been taking periodic snapshots and need to find "the one from before yesterday's deploy."' This implies when to use it, but does not explicitly state when not to use it or compare to alternatives like snapshot_diff for comparing snapshots.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It explains parameters but does not disclose side effects (e.g., overwriting existing bindings), required permissions, or potential impacts beyond the operation itself.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is structured with a brief summary followed by a parameter list, but it is somewhat verbose with full example URIs. It could be more concise while retaining clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 5 parameters and 3 required, the description covers them well. However, there is no mention of the output schema (e.g., success/error responses) or any post-condition behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides no descriptions (0% coverage), but the description compensates with detailed explanations for each parameter, including examples for table, aor, contact, and default values for expires and q.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Add a contact binding') and the target resource ('user location table'). The verb-resource pair is specific and distinguishes it from sibling tools like ul_remove_contact and ul_show_contacts.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no explicit guidance on when to use this tool vs alternatives or prerequisites. While the purpose is clear, there is no mention of when not to use it or compare to other tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It truthfully states it 'lists' calls, implying a read operation, but does not mention permissions, side effects, or performance characteristics.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence of 10 words, front-loading the core purpose with no extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple nature of the tool (list all queued calls) and the presence of an output schema, the description is adequate. However, it could provide more context, such as what 'queued' means or if the list is ordered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters, so schema coverage is 100%. The description does not need to add parameter information, earning a baseline of 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: listing all calls currently queued in the call center. The verb 'List' and resource 'calls queued' are specific, and it distinguishes from sibling tools like 'cc_list_agents' and 'cc_status'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for viewing queued calls but provides no explicit guidance on when to use this tool versus alternatives, nor any conditions or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description carries full burden. It likens the tool to kubectl explain, implying a read-only lookup, but does not explicitly state 'read-only' or 'no side effects.' Mentions version-rename history but lacks details on behavior like auth or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Efficiently structured: introduction, bulleted examples, parameter description. No redundant sentences; every part adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given single parameter and presence of output schema, the description fully covers what the agent needs: what the tool does and how to specify the topic. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description adds essential meaning. Provides detailed examples of topic syntax (module overview, modparam, function, etc.), adding clarity beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states that the tool explains OpenSIPS directives, modules, functions, or pseudo-variables. Provides concrete examples that distinguish it from siblings like cfg_explain_route.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like cfg_explain_route. The examples imply broad applicability but do not define exclusions or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses output format, allowlist requirement for out_dir, and optional label. However, does not explicitly state non-destructive nature (likely read-only) or error handling. No annotations to supplement.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: one clear sentence plus bulleted parameter descriptions. No fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, parameters, output, and constraints. With an output schema present, the description is sufficient for a simple backup tool. Minor omission: does not explain return value beyond file creation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds significant meaning beyond schema: lists permissible values for 'table', describes 'out_dir' constraint (safe allowlist), and explains 'label' as filename tag. Compensates for 0% schema description coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action (dump), resource (table), and output (timestamped JSON file). Lists supported tables, distinguishing it from siblings like db_table_restore and db_table_list_backups.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies use for backups but lacks explicit guidance on when to use vs. siblings (e.g., restore, list). No mention of alternatives or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It explains the dry_run parameter's effect ('Returns the plan without writing') but does not mention potential side effects like overwriting existing sets, required permissions, or persistence beyond testing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear first line, usage scenario, and parameter list. It is concise but the parameter section is slightly verbose for a tool description, though it adds necessary detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the tool has an output schema, the description does not mention what the tool returns (e.g., the plan or success message). It also does not explain system impact, making it only partially complete for a provisioning tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description compensates by explaining each parameter: set_id's default, destinations with example SIP URIs, dry_run's behavior, and socket's purpose. However, 'per row' is ambiguous and could be clearer.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool 'Provision a dispatcher set with a handful of test destinations', using a specific verb and resource. It distinguishes itself from siblings like 'dispatcher_add' by focusing on bulk test setup for empty dispatcher tables.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly mentions when to use the tool: 'when you have a scenario template that uses ds_select_dst(...) but the dispatcher table is empty.' It provides context but does not contrast with alternatives like 'dispatcher_add' or mention when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, leaving the description to cover behavioral traits. It indicates a read operation (Retrieve) but does not explicitly state read-only status, permissions, or data freshness. Adequate for a simple tool but missing explicit context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. The key action and output are front-loaded, making it immediately understandable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and an output schema, the description is largely complete. It could briefly mention that the statistics are real-time or cached, but it adequately describes purpose and output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, and schema description coverage is 100% (trivial). The description adds value by specifying the return content (method distribution, totals) beyond the schema, which only defines an empty object.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Retrieve', the resource 'Homer capture statistics', and specifies the output includes method distribution and totals. It is distinct from sibling homer tools like homer_search_calls or homer_health.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use when Homer capture statistics are needed, but does not provide explicit guidance on when to use this tool versus other Homer-related tools, nor does it mention any prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavioral traits. It only mentions tolerating '.so' suffix as an input flexibility. It does not specify whether the operation is read-only, idempotent, or what happens for invalid modules, leaving significant behavioral gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two short sentences plus a clear parameter note in a code block. Every sentence adds value, and the structure is well-organized with front-loaded purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and likely an output schema (not shown), the description covers the input semantics and use case. It lacks explanation of what 'dashboard categories' represent or how the output is structured, but the purpose and basic behavior are clear.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has a single array parameter 'modules' with no description (0% coverage). The description compensates by explaining that module names can have optional '.so' suffix. This adds meaningful semantic information beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Report' and the resource 'which dashboard categories' for a given module set. It distinguishes from sibling tools like observability_generate_bundle by calling itself a 'preview before calling the bundle generator'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Useful for a what would I get? preview before calling the bundle generator', which provides clear usage context. It does not list exclusions or compare to other siblings like observability_list_panel_categories, but the guidance is direct and actionable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description indicates read-only operation ('list') but does not disclose potential side effects, permissions, or performance impact.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with no unnecessary words, front-loaded with the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While output schema exists and description mentions what is returned, it does not explain 'pipes' or counter format; still adequate for a simple list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist; description adds no parameter info but schema coverage is 100% trivially, so baseline 4 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb (list) and resource (rate limiting pipes) with specific detail (current counters), distinguishing it from sibling tool ratelimit_reset.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use vs alternatives like ratelimit_reset; usage is implied from the read-only nature.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description must carry full burden. It lists returned counters and internal command, but doesn't disclose side effects, permissions, or prerequisites (e.g., module loaded). Adequate but not thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded, no fluff. Bullet-pointed return categories for clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With output schema present, description is nearly complete: it explains purpose, return values, and internal mechanism. Could mention read-only nature or module dependency.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters in input schema (0 params), baseline 4. Description adds no parameter info but does list return fields, which is helpful.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool pulls TLS-specific statistics from the tls_mgm module, listing exact counters returned. It distinguishes from sibling tools like get_statistics and other TLS tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this vs alternatives like get_statistics. Implied that it's for TLS stats only, but lacks direct comparison or exclusion statements.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It correctly indicates it is a read operation and returns three datasets, but does not disclose auth needs, rate limits, or error handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the purpose, no unnecessary words. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple parameterless read tool, the description fully explains the output (three datasets) and is complete given the presence of an output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameters, so the description adds no parameter info beyond what the schema provides. Baseline of 4 applies as per criteria.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'combined call center status: queue, agents, and flows', distinguishing it from sibling tools like cc_list_agents and cc_list_queue.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for a complete overview but does not explicitly state when to use this tool versus separate queue/agent tools or provide exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes analysis behavior (identifies components, methods, functions, conditions, flow) beyond schema. No annotations, but description suffices for a read-only analysis tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Some redundancy between the first line and the subsequent paragraph, but front-loaded with purpose. A bit verbose for a single-parameter tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers input and behavior well, and output schema exists (though not shown). For a single-param tool, it is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema coverage, description fully explains route_code as the route block body and gives an example, adding significant value over the schema's minimal 'Route Code' label.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it explains an OpenSIPS route block by identifying components like SIP methods, module functions, conditions, and flow, distinguishing it from sibling tools like cfg_explain or cfg_lint.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage for analyzing route blocks but does not explicitly mention when to use vs alternatives like cfg_explain for full config or cfg_lint for validation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description implies read-only generation (returns strings) but does not explicitly state side effects, permissions, or safety. The behavioral traits are somewhat clear from context but not fully disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with a clear purpose statement, followed by structured parameter descriptions. Each sentence adds value without redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the description adequately covers parameters and output purpose. Minor gaps exist (e.g., error handling, case sensitivity) but overall it is sufficiently complete for a generation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description adds concrete examples and allowed values for scenario, explains template_params as 'Structural/Jinja2 template knobs', and site_params with a dict example, providing essential meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Render' and the resource 'm4 template pair (main + local) for a scenario', distinguishing it from sibling tools like cfg_generate and cfg_generate_iterative which likely generate non-m4 outputs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides workflow guidance (persist to specific files then preprocess with m4) but lacks explicit when-to-use vs alternatives like cfg_generate or cfg_generate_iterative, and does not state when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Despite no annotations, the description reveals key behaviors: automatic backup of current contents, validation behavior (including that validation failure does not undo the restore), and that results are surfaced. The only missing detail is potential side effects like permissions or locking.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description uses clear sections and bullet points, but the initial paragraph is dense. It could be shortened while retaining key information, but overall it is well-structured and not overly verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the tool's behavior and parameters but lacks details about the output schema (only mentions 'result is surfaced'). Given the tool has an output schema, the agent needs to know what the response contains (e.g., success status, validation errors) to act on the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema coverage, the description fully explains all three parameters: backup_path (absolute path to backup), target_path (absolute path in allowlisted dir), validate_after (default True, runs opensips -C). This adds critical context beyond the schema's type-only definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action ('Restore backup_path to target_path and re-validate') and distinguishes it from sibling config tools by noting the automatic backup before overwriting, making it uniquely reversible.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for config rollback but does not explicitly state when to use this tool over alternatives like cfg_backup or cfg_validate. No guidance on excluded scenarios or prerequisites beyond target path allowlisting.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries burden. It states what is returned (endpoint configs and summary) but doesn't mention side effects, safety, or authentication. Adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, purpose first, then specifics. No wasted words, highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and presence of output schema, description fully explains what the tool returns. Completes the picture for a simple overview tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters, schema coverage 100%. Description adds no param info beyond schema, but baseline for 0 params is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool shows an overview of ecosystem component URLs and available tools, specifies it returns endpoint configuration for Homer, RTPEngine, and Control Panel. Distinct from sibling 'ecosystem_health'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or alternatives mentioned, but description implies it's for initial overview. Lacks guidance on when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description carries the burden. It correctly indicates a read operation ('Get'), but does not disclose any additional behavioral traits such as side effects or prerequisites. Adequate for a simple status tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence of 8 words, front-loaded with the key action and result. No unnecessary information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is complete for a simple no-parameter tool with an output schema. It states the purpose adequately; the output schema covers return values as per guidelines.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters in the input schema, and schema description coverage is 100% trivially. Baseline is 4 for 0 parameters, and the description does not need to add parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Get overall fail2ban status including list of active jails.', specifying the verb and resource. It distinguishes from siblings like fail2ban_jail_status which focus on specific jails.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like fail2ban_jail_status. The purpose is implied by the name and description, but no direct comparison or exclusions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It explains what files are created and optional MI commands, but lacks disclosure on potential overwrites, authorization needs, or error conditions. The behavior is partially transparent but incomplete for a generation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured. The first sentence states the core purpose, followed by a list of output files, then a clear 'Parameters' section with bullet points. No wasted sentences, and critical info is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that an output schema exists (context signal) and parameters are well-documented, the description is mostly complete. It lacks details on return value and error conditions, but for a generation tool with detailed params, it is adequate. Not perfect, hence 4.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description fully compensates. It explains each parameter: name must be a valid C identifier, description is short text, mi_commands_json and params_json have clear structure with examples. This adds significant value beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Generate a complete OpenSIPS C module skeleton.' It specifies the exact outputs (C source, header, Makefile, README, optional MI commands) and distinguishes itself from sibling tools by focusing on module creation from scratch.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly guide when to use this tool vs alternatives. There are sibling tools like module_add_mi_command and module_compile for subsequent steps, but no mention of when to use scaffolding vs those. The context implies it's for new modules, but no when-not-to or alternative references.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description carries the full burden. It explains the internal process: retrieves all TCP/TLS via list_tcp_conns then filters to TLS. It also lists the fields returned (IP:port, state, protocol, alias). This is clear and honest.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise: two sentences for purpose and then a bulleted list of details. Every part adds value, and the structure is front-loaded with the main action. No filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that there are no parameters, an output schema exists (though not shown), and the description explains what the tool does and what information it returns, it is complete. The tool is simple, and the description covers all needed context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameters, so the baseline for 0 params is 4. The description does not need to add parameter semantics because there are none. It correctly indicates no input is required.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with 'List active TLS connections with details,' which clearly states the verb (List) and the resource (active TLS connections). It distinguishes itself from siblings like get_tcp_connections by specifying it only shows TLS connections, and it mentions filtering from TCP connections.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. It does not mention when not to use it or what other tools are available (e.g., tls_list, get_tcp_connections). The description only states what it does.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description indicates it reads files from OUT_DIR and produces a table, implying no side effects, but doesn't disclose permissions, error conditions, or if any modifications occur. Reasonable for a summary tool but could be more explicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first defines the action, second provides use case and output example. No wasted words, front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given one parameter, no annotations, and an output schema (exists but not shown), the description adequately explains the tool's purpose and output. Could specify that it is read-only and what 'artefact' means.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter (out_dir) with default. Description links it to the directory of artefacts ('every bench artefact in OUT_DIR'), adding meaning beyond the schema which has 0% description coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it aggregates bench artefacts into a summary table, listing columns. Distinct from sibling tools that run benchmarks (bench_quick_invite, bench_quick_register).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Useful at the end of a tuning run' and shows what the output contains, but does not explicitly mention when not to use or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Discloses scanning for specific anomalies and parameter defaults but does not explicitly state read-only nature or potential side effects. Adequate but could be more explicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Very concise: a one-line summary followed by clear parameter list. No redundant or extraneous information. Well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, parameters, and usage constraints. Output schema exists but description does not detail return values, which is acceptable. Adequate for a scanning tool with moderate complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage 0%, so description adds meaning: explains ISO timestamps, window cap, defaults, and anomaly detection specifics. Adds value beyond schema titles and defaults.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Scan the acc table for gaps, duplicates, and duration anomalies.' Specifies verb (scan), resource (acc table), and anomalies. Differentiates from siblings like cdr_summary and acc_query_cdrs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Mentions date window capped at 31 days and advises longer ranges use warehouse/BI tool. Provides explicit context for when to use but does not name alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must disclose behavior. It explains what it reports (missing modules, deprecations, defaults) but does not specify if it is read-only or if it requires any permissions. It implies a non-destructive check but lacks explicit assurance.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is brief and well-structured: a clear intro, bullet points of checks, and a usage note. Every sentence adds value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (not shown but known from context), the description appropriately focuses on inputs and behavior. It covers the key aspects of what the tool does, though it could mention if the config must be valid or if there are limitations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description compensates by explaining each parameter: config_content is the text to check, target_version is the target version. This adds meaningful context beyond the schema's title and type.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool checks a config for incompatibilities with a target OpenSIPS version, listing specific items (non-existent modules, deprecated parameters, defaults). It distinguishes itself from sibling tool cfg_migrate_plan.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description advises pairing with cfg_migrate_plan and suggests running compat first. This provides usage order guidance, though it does not explicitly list when not to use it or alternatives beyond that pairing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes non-destructive behavior ('without rewriting'), outputs as ordered hops, automatic rewrites, manual steps, and warnings. With no annotations, provides solid behavioral context, though omits permissions or error scenarios.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Short, front-loaded description with concise bullet-like parameter details. No superfluous text; every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Describes what the plan includes (hops, rewrites, manual steps, warnings), fitting for a planning tool. Output schema exists, so return format is covered elsewhere. Could specify output format, but acceptable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All three parameters (config_content, from_version, to_version) are explained with clear descriptions and examples (e.g., version formats). Since schema coverage is 0%, description fully compensates by adding meaning beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it reports what a version migration would do without rewriting, using the strong analogy 'Terraform-plan equivalent'. Distinguishes from sibling 'cfg_migrate' which does the actual migration.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implied use case (preview migration impact), but no explicit when-not-to-use or comparison to alternatives like 'cfg_dry_run'. Missing guidance on prerequisites or condition checks.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must carry behavioral info. It discloses directory validation (rejects system directories) and optional m4 run. However, it does not explicitly mention file overwriting behavior or idempotency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Structured with intro, file details, validation warning, and parameter list. Slightly verbose for the parameter descriptions, but overall concise and front-loaded with key action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, files, optional behavior, and directory constraints. Output schema exists but not shown, so return value is not described. Still adequate for a save tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but description provides detailed parameter descriptions (out_dir created if missing, main_m4/local_m4 contents, write_preprocessed triggers m4 run). This adds meaningful semantic context beyond titles and types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool persists m4 pair to disk and optionally writes preprocessed cfg, with specific file paths. This distinguishes it from other cfg_* tools like cfg_diff or cfg_start_session.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use vs alternatives. It describes functionality well but does not contrast with sibling tools like cfg_preprocess or cfg_start_session.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries full burden. It discloses that it executes a specific command ('docker compose ps') and returns output, implying a read-only operation. No side effects or permissions are mentioned, but for a status check, this is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the purpose and followed by implementation detail. No redundant information; every sentence is essential.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple zero-parameter tool with an output schema, the description provides adequate context. It states what the tool does and how. The mention of 'returns the output' could be more specific about format, but the output schema likely covers that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, and schema coverage is 100%. The description adds value by specifying 'running' Docker Compose services, implying the status is limited to active services. Baseline for zero params is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Show') and resource ('status of running Docker Compose services'), and differentiates from sibling docker tools (e.g., docker_logs, docker_restart) by specifying it returns the output of 'docker compose ps'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use vs alternatives. However, given the tool's simplicity and zero parameters, the purpose is self-explanatory. It does not mention when not to use or compare to other docker tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses that the tool generates YAML files for datasource and dashboard provider, and explains the deployment effect. It does not mention any destructive actions, auth requirements, or side effects, but the tool appears non-destructive and the behavior is clearly described. The description provides sufficient behavioral context for an AI agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the primary purpose in the first line, followed by a concise paragraph explaining deployment behavior, then a clean list of parameters. Every sentence adds value without redundancy. The docstring-style 'Parameters' section is slightly verbose given the schema, but overall it is efficient and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description explains the tool's output effect (YAML files for provisioning) and deployment steps. Although it does not describe the return value of the tool itself, the context signals indicate an output schema exists, so the description is not required to detail return format. Given the parameter count and simplicity of the tool, the description is adequately complete for an AI agent to understand usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 3 parameters with defaults and titles but no property descriptions (schema description coverage 0%). The description compensates by listing each parameter with a brief explanation: 'URL Grafana should use to query Prometheus,' 'Grafana folder name dashboards land in,' and 'Filesystem path inside the Grafana container where dashboard JSONs are mounted.' This adds meaningful context beyond the schema's bare property names and defaults.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Generate Grafana provisioning YAMLs (datasource + dashboard provider).' It specifies the verb 'Generate' and the resource 'Grafana provisioning YAMLs,' and further explains the purpose by describing the files' behavior when dropped into Grafana's provisioning directory. This clearly distinguishes it from sibling observability tools that generate dashboards or bundles.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains how to deploy the generated files (drop into /etc/grafana/provisioning/) and what happens (auto-register datasource, auto-load dashboards). However, it does not explicitly state when to use this tool versus alternatives like observability_generate_dashboards or observability_generate_bundle. It lacks guidance on prerequisites or scenarios where this tool is preferred.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
In the absence of annotations, the description discloses key behaviors: each category produces one dashboard JSON when its trigger modules are present, 'overview' is always emitted, and the exact return format is given. No contradictions found.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: one sentence for purpose, one for details, one for return format. No redundancy, all information is relevant and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters, straightforward return structure), the description provides sufficient context. Could optionally mention ordering or usage with other observability tools, but not necessary for basic understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With zero parameters and 100% schema coverage, the description adds value by explaining the contextual purpose of the return values (categories with fields like name, title, description, etc.), which goes beyond the empty schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List the Grafana panel categories the dashboard generator can emit,' with a specific verb and resource. It distinguishes itself by focusing on panel categories for dashboard generation, not generic categories like observability_categories_for_modules.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving panel categories needed to generate dashboards, but does not explicitly state when to use this tool versus alternatives like observability_categories_for_modules. The context is implied but not definitive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It clearly states what the tool does (computes deltas, flags changes) and the default behavior of omitting unchanged keys. It does not contradict any annotations, as there are none.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: a brief intro followed by a clear parameter list. Every sentence adds value, and the format is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity and the presence of an output schema (though not detailed), the description is fairly complete. It explains inputs and behavior, but lacks explicit details about the output format, which is a minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description must compensate. It explains all three parameters (before_path, after_path, show_unchanged) with their meaning and default value, adding significant value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the tool's purpose: 'Diff two snapshots and surface the deltas.' It lists specific deltas computed (per-stat diffs, process appearance/disappearance, state changes), distinguishing it from sibling tools like 'snapshot_capture' and 'snapshot_list'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage (when you need to compare snapshots) but does not explicitly guide when to use this tool vs alternatives or when not to use it. No exclusions or prerequisites are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It explains it queries TLS domains, parses PEM, and returns a summary. It does not mention side effects or read-only nature, but it's implied it's a read operation. Could add 'no changes made' for clarity.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with purpose, followed by bullet-like details. Every sentence adds value, no fluff. Well-structured and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Output schema exists but description still summarizes output fields. Covers domain, expiry, days, status. Missing prerequisites (e.g., TLS configured), but overall adequate for a simple scan tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has zero parameters (100% coverage), so description need not add parameter info. Baseline for no params is 4. Description correctly omits parameter details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it performs a bulk certificate expiry scan across all configured TLS domains. Differentiates from sibling 'tls_check_expiry' by checking all domains, listing output fields and status categories.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
While it does not explicitly state when to use vs alternatives, the implication is clear that this is for a comprehensive scan of all domains, contrasting with a potential single-domain check. However, no explicit when-not or alternative tool mention.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of behavioral disclosure. It explains that it pulls specific counters and provides severity assessment, but does not state whether it is read-only, mutates state, or requires permissions. The description is adequate for a read operation but lacks explicit safety traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise with only two sentences, front-loaded with the core purpose, and every sentence adds value. It efficiently communicates the tool's functionality and output structure without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no parameters and a clear output schema, the description covers the key aspects: what it does, how it works (pulling from get_statistics), and what it returns (error type, count, percentage, severity). It is complete enough for an agent to select and invoke correctly, though the output schema might provide further structure.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the description does not need to add parameter details. The input schema coverage is 100% by default. The description adds meaning by hinting at what the output contains (error type, count, percentage, severity), which is sufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description specifies a 'focused view of TLS error statistics with severity assessment', clearly stating the verb ('view'), resource ('TLS error statistics'), and additional value ('severity assessment'). It distinguishes itself from sibling tools like 'get_statistics' and 'tls_statistics' by noting it only returns error-related counters with severity levels.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for obtaining TLS error statistics with severity, which is clear context for use. However, it does not explicitly state when not to use it (e.g., for general TLS statistics use 'tls_statistics' or 'get_statistics'). This is a minor gap given the sibling context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden. It correctly identifies the tool as a list operation (read-only) but does not disclose details like output format, pagination, or authentication needs. The presence of an output schema mitigates some gaps, but the description itself offers minimal behavioral context beyond the purpose.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the purpose, followed by a usage note and parameter documentation. It is concise with no wasted words, and the parameter list is well structured with examples.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (list with filters) and the existence of an output schema, the description covers purpose, use case, and parameters adequately. Minor omissions like what the output contains are addressed by the output schema, so it is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains both parameters clearly: version with an example ('3.6') and category with a list of category names. This adds significant meaning over the schema's bare titles.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'List modules from the OpenSIPS module catalog', which is a specific verb and resource. It also explains the use case by the wizard and LLMs to check module availability, clearly distinguishing from sibling tools like cfg_add_module or cfg_list_versions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear context for use: 'by the wizard and by LLMs that need to check whether a module is available in a given version.' However, it does not explicitly mention when not to use it or suggest alternatives, though siblings are distinct.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description correctly indicates read-only nature by stating it returns data, but lacks detail on side effects or limitations beyond listing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences with front-loaded verb and resource, no wasted words; efficiently covers purpose, return value, and usage guidance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter listing tool with output schema, description fully covers purpose, usage, and return value; no gaps given simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 0 parameters, so description cannot add param info; baseline 4 since no parameters exist.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it lists supported OpenSIPS versions and migration hops, distinguishing it from sibling tools like cfg_migrate and cfg_migrate_plan.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly instructs to use this before cfg_migrate to confirm source/target support, providing clear when-to-use context; could mention when not to use but adequate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses critical constraints: commands must be registered in MI_COMMANDS and per-command permission is enforced, preventing bypass of mi.write controls. This provides strong transparency, though it could mention error behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One introductory sentence followed by a clear parameter list; no redundant or extraneous content. Efficient and scannable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, parameters, and key behavioral constraints. With an output schema present, return details are handled separately. Could mention error conditions or output format hints, but overall adequately complete for a command-sending tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must explain all parameters. It defines each parameter and adds crucial context for mi_cmd (registration, permissions) and mi_params (optional JSON string). This fully compensates for the schema's lack of descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with a clear verb+resource: 'Send an MI command to a specific node in the cluster.' It uniquely identifies the remote execution scope, distinguishing it from local MI command tools like mi_execute.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage through parameter context (requires cluster_id, node_id) and notes permission enforcement, but does not explicitly state when to prefer this tool over alternatives like mi_execute or other cluster tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It correctly describes a read-only listing operation with no side effects. No additional behavioral traits (e.g., auth, rate limits) are needed for such a simple tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: first states the purpose, second describes the output. No unnecessary words, well-structured, and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters, output schema exists), the description adequately covers the context. It explains what the output contains and that it reflects advertised event types.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so schema coverage is 100%. The description adds no parameter info, but that is fine since none exist. The baseline for no params is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists all event types advertised by the running OpenSIPS, with details on the output. It uses a specific verb and resource, and distinguishes from siblings like event_subscribe and event_list_subscribers.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide explicit when-to-use or when-not-to-use guidance, nor does it reference alternative tools. However, the purpose is straightforward, so minimal guidance is arguably acceptable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses dry_run default behavior and a count cap of 500, but lacks details on side effects, permission requirements, reversibility, or behavior on failure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, front-loaded with purpose, and uses a clear parameter list. Every sentence adds value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters and an output schema (not described), the description covers input details well but omits output structure and relationships to subscriber management tools. Still, it's sufficiently complete for the task.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description explains every parameter in detail: domain validation, count cap, username/password prefix patterns, and dry_run default, adding significant meaning beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool generates N test subscribers with predictable usernames and passwords, and explicitly mentions its use for smoke-testing REGISTER paths and synthetic load, distinguishing it from sibling tools like subscriber_create or subscriber_bulk_import.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear usage context ('useful for smoke-testing REGISTER paths and synthetic load against a fresh deployment') but does not explicitly mention when not to use this tool or point to alternatives among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It only states what the tool does but does not disclose any behavioral traits (e.g., return format, performance implications). The output schema exists but is not detailed in the description.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence (9 words) with no waste, front-loaded with the verb and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (zero parameters, output schema exists), the description is complete. It tells what the tool does, and the output schema covers the return format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so the baseline is 4. The description adds no additional parameter information, but none is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Show how long the OpenSIPS instance has been running' uses a specific verb ('Show') and a clear resource ('uptime of OpenSIPS instance'). It distinguishes itself from sibling tools as no other tool mentions uptime.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Although no explicit when-to-use or alternatives are provided, the tool is unique and its purpose is clear. The context implies that it should be used when the agent needs to know the system uptime.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It explains return format (pcap bytes, optionally base64-encoded) and use in Wireshark. Lacks info on side effects, auth needs, or errors.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Four short sentences plus parameter list. No fluff, front-loaded with main action, then details. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, usage, output format, and parameter semantics. Output schema exists, so return values are covered. Minor gap: doesn't mention prerequisites like call existence or error scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 0% schema coverage, description explains each parameter: callid as exact SIP Call-ID, from_ts/to_ts as Unix millisecond time window. Adds significant meaning beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool exports captured SIP messages as pcap for a given callid. Distinguishes from sibling tools like homer_get_call_flow by specifying use case for raw bytes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says to use when SIP ladder is not enough for raw byte inspection. Implies alternative, though doesn't name homer_get_call_flow directly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully bears the burden. It discloses that the tool returns captured SIP messages across proxies in timestamp order, explains the internal reconstruction mechanism, and notes strict time-window requirements. No contradictions with annotations (none exist).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is well-structured, starting with the core purpose, then mechanism, then a bulleted parameter list. It is front-loaded and concise, with no redundant sentences. Could slightly trim the endpoint explanation, but it is effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (SIP transaction fetch), the presence of an output schema (so return values are handled), and no annotations, the description covers purpose, mechanism, parameter details, and sibling differentiation. It omits rate limits or authentication notes, but is otherwise complete for an agent to select and invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It adds clear semantics: callid is 'Exact SIP Call-ID', from_ts and to_ts are 'Unix milliseconds' with the note that 'Homer is strict — the window must bracket the call start'. This goes beyond schema types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description explicitly states 'Fetch the full SIP transaction ladder for a single Call-ID', using a specific verb and resource. It distinguishes from sibling homer_search_calls_v7 by explaining the mechanism (parameter flag and endpoint reuse), leaving no ambiguity about the tool's unique function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description explains that it uses the same endpoint as homer_search_calls_v7 but reconstructs the transaction via a parameter, providing clear context for when to choose this tool. It doesn't explicitly list when not to use it, but the alternative is implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states that it returns statistics, implying a read-only operation, but does not explicitly mention non-destructive behavior, permissions, or performance impact. This is adequate but lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long with no unnecessary words. The first sentence states the core function, and the second provides usage context. It is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and an output schema exists, the description only needs to explain what the tool returns and when to use it. It does this completely, covering purpose and utility.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters in the input schema, so the description has no need to add parameter meaning. The baseline for zero parameters is 4, and the description meets it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it returns 'nathelper runtime statistics' and specifies 'ping counts, failure rate', which is a specific verb+resource combination. This distinguishes it from sibling tools like nathelper_enable_ping and other stats tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides practical use cases: 'diagnosing NAT keepalive issues and sizing ping load'. While it doesn't explicitly state when not to use it or list alternatives, the context is sufficient for an agent to decide when to invoke this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses the generation process, required validation, and output structure ({dashboards: ...} plus module list). It does not mention side effects, rate limits, or destructive actions, but for a generation tool this is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a summary, process explanation, and parameter list. It is comprehensive without being verbose, though the parameter list could be slightly more concise. Still, it earns its length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (4 parameters, optional, no enums, with output schema), the description fully covers necessary information: input requirements, behavior, and return format. It references related tools (cfg_list_scenarios, cfg_generate) for context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description provides detailed parameter explanations: scenario (from cfg_list_scenarios), params (same as cfg_generate), deployment_name (slug with default), extra_tags (additional tags). This adds significant meaning beyond the schema's type-only information.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Render a scenario and synthesize matching Grafana dashboards.' It specifies the verb (render/synthesize) and resource (scenario and dashboards), and distinguishes from siblings by focusing on scenario rendering and panel category production.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the process (validates params, parses modules, produces dashboards per category) but does not explicitly state when to use this tool versus alternatives like observability_generate_dashboards_from_cfg. Usage is implied through parameter references (scenario from cfg_list_scenarios, params from cfg_generate), but no when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Transparent about generating a docker-compose snippet, with no annotations. Does not mention destructive effects, but as a code generator it is inherently safe and read-only.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with a brief header, assumption note, and parameter list. Every sentence adds value; no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers functionality and parameters well. Since an output schema exists, return value explanation is not needed. Could mention output format but optional.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 0% schema description coverage, the description provides clear explanations for all 5 parameters, explaining their purposes (container names, ports, password, network) and defaults.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it generates a docker-compose snippet for a Prometheus + Grafana side-car, distinguishing it from sibling observability tools that generate bundles or dashboards.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides assumptions about network setup but does not explicitly state when to use this tool vs alternatives like observability_generate_bundle or observability_generate_dashboards.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must cover behaviors. It discloses idempotency and warnings about port conflicts, but does not state what the tool returns (the modified config) or any side effects like overwriting existing parameters. This is a gap for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear purpose statement, a numbered list of actions, and a parameter section. Every sentence serves a purpose; no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (config injection with multiple steps) and lack of annotations, the description covers purpose, parameters, idempotency, and warnings. It assumes knowledge of OpenSIPS but is sufficient for the target audience. The presence of an output schema partially covers return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description adds clear explanations for both parameters: config_content is existing config text, prom_port is the TCP port with default 8888. This fully compensates for the schema's lack of descriptions and adds usage context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool adds prometheus.so and httpd.so to an OpenSIPS config, listing the exact lines. It distinguishes from sibling tools like cfg_add_module and other observability tools by focusing specifically on enabling Prometheus monitoring.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains that most scenarios don't load prometheus.so by default, implying use when enabling Prometheus. It mentions idempotency and return warnings, providing context. However, it doesn't explicitly compare to alternatives like cfg_edit for general config changes.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears the full burden. It describes a read operation without mentioning any side effects or prerequisites. For a simple read, this is adequate but not exceptional.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is concise and front-loaded. Every word contributes to meaning, with no wasted content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given tool simplicity, presence of output schema, and no parameters, the description fully covers what an agent needs to know. It is complete for the task.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameters and 100% schema description coverage. The description correctly implies no parameters are needed. It adds no extra parameter info, but none is required.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action ('Show'), the resource ('version string'), and the context ('connected OpenSIPS instance'). It is specific and unambiguous, leaving no room for confusion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
While not explicitly stating when not to use it, the tool's purpose is self-evident. It is a simple informational query, and the context makes its usage clear. A small deduction for lack of explicit guidance on alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should disclose behavioral traits. It indicates a read-only summary but does not mention side effects, permissions, rate limits, or cost. It adds basic transparency about what data is combined but is insufficient for full behavioral understanding.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is four sentences, front-loaded with the main purpose, and every sentence adds value. No redundant or vague language.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple summary tool, the description is largely complete: it explains the combination of sources and the returned fields. However, it could mention more about when to use this instead of other stats tools or note potential limitations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, so the description cannot add parameter semantics beyond the schema. However, it compensates by detailing the output composition and values, which is valuable context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool provides a quick summary of system pain points by combining multiple data sources. It specifies the exact values returned and distinguishes itself from siblings by being a high-level diagnostic entry point.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when an operator or LLM needs to decide whether to dig deeper, but does not explicitly state when to use versus alternatives like get_statistics or perf_memory_report. The context is clear but lacks explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description carries full burden. It accurately describes a simple read operation without side effects, but could mention that it requires no parameters and returns a list.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no unnecessary words, perfectly front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity (no parameters, output schema exists), the description fully covers the tool's purpose and behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With zero parameters, the baseline is 4. The description adds no parameter info, but none is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('List') and resource ('IP addresses currently blocked by the Pike anti-flood module'), clearly distinguishing it from the sibling 'pike_unblock_ip' which performs the opposite action.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for listing blocked IPs but does not explicitly state when to use this tool versus alternatives like 'pike_unblock_ip' or other related tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses backup location (next to original or backup_dir), allowlisted directories for config_path, and naming format. No annotations provided, so description carries full burden; it covers key behaviors without contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise with a clear summary and bulleted parameter list. Every sentence adds value, no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Explains return value (backup path for rollback), covers parameters and constraints. Could mention writability of backup_dir, but sufficient for a simple backup tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage; description compensates by explaining config_path must be absolute and in allowlisted directories, and backup_dir is optional. Adds meaningful context beyond schema types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it copies a config file to a timestamped backup, with explicit naming convention and return value. Distinguishes from sibling cfg_rollback by mentioning its output can be used for rollback.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides context that the backup path is intended for cfg_rollback, implying usage. Does not explicitly state when not to use or list alternatives, but the purpose is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, but description declares 'write nothing' and 'safely iterate', indicating safe, read-only operation. Adds value beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded summary, then usage guidance, then parameter details. Concise, well-organized, no redundant sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool has two parameters and output schema exists. Description covers parameter semantics and usage. Could mention output but output schema handles that. Adequate given context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but description adds clear explanations for main_m4 and local_m4 as contents of specific files, compensating for lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool preprocesses and validates config without writing, distinct from sibling tools like cfg_validate or cfg_preprocess due to 'dry run' emphasis.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says use as tight feedback loop between successive renders, implying when to use. Does not explicitly contrast with siblings but usage context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries the full burden. It correctly indicates a read-only listing operation, which is transparent. However, it does not mention output format or any side effects, but with an output schema available, this is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. It is maximally concise for the information it conveys.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (no parameters, output schema exists), the description is adequate. It could elaborate on what 'locally-served' means, but it is largely complete for an agent to understand its purpose.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the description does not need to add parameter-level meaning. The baseline for 0 parameters is 4, and the description does not mislead.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses the specific verb 'List' and clearly identifies the resource as 'all locally-served SIP domains', which distinguishes it from sibling tools like domain_add or domain_delete.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly states the context (listing domains) and implicitly indicates when to use it. However, it does not explicitly contrast with alternative tools or mention any prerequisites, which would be ideal.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions ms-precision and source-IP filter but does not disclose read-only nature, error handling, rate limits, or other behavioral traits. Adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is well-structured with a brief introductory line followed by a clear parameter list. One sentence could be removed (e.g., the final line about the parameter list is slightly redundant with schema) but overall efficient and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 7 parameters, 2 required, and an output schema present, the description covers core aspects: parameter semantics and usage context. Could elaborate on return value structure, but output schema exists to handle that. Missing a note on rate limiting or pagination, but not critical for a search tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the parameter section in the description adds detailed meaning: from_ts/to_ts described as Unix ms, from_user/to_user as substring match, callid as exact, source_ip validated, limit capped at 500. This fully compensates for the schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Homer 7 native search with ms-precision timestamps + source-IP filter' and explicitly distinguishes itself from sibling homer_search_calls which uses ISO strings, making the specific resource and verb unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description explicitly says 'Prefer this when you are chasing a tight time window or want to filter by source IP' and references the alternative homer_search_calls, providing clear guidance on when to use this tool vs. its counterpart.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses that by default inject_prometheus adds prometheus.so and httpd.so to the cfg, and provides defaults for ports and host. It does not discuss file overwrite behavior or idempotency, but covers key behavioral changes sufficiently.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a summary, output structure code block, a note on category selection, and parameter list. It is informative without being overly verbose, though could be slightly more concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (7 parameters, output schema present), the description adequately covers the bundle contents, default behaviors, and parameter meanings. It does not mention error handling or prerequisites beyond having a scenario, but is largely complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description compensates with a detailed parameter section, explaining each parameter's purpose, default values, and usage context (e.g., 'inject_prometheus: Add prometheus.so + httpd.so to the cfg if absent. Default True. Set False if you want to wire the exporter yourself.').
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Build a full deployment bundle from a scenario.' It details the output structure (opensips.cfg, monitoring/ with docker-compose, dashboards, README) and distinguishes from siblings like observability_generate_dashboards by being the comprehensive bundle that combines cfg generation with monitoring setup.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies it's used when you need a full deployment bundle, and contrasts with sibling tools by being all-encompassing. However, it does not explicitly provide when-not or list alternatives, leaving some inference to the agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description discloses key behaviors: dependency order, module skipping, partial deployment success, stop_on_error behavior, and delay rationale. It lacks explicit mention of output format (though output schema exists) or idempotency/safety.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is fairly long but well-structured with parameter explanations and rationale. Every sentence adds value; no redundancy. Could be slightly more concise, but efficient overall.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (3 params, output schema exists, many siblings), the description covers the tool's behavior, parameters, and edge cases thoroughly. It lacks explicit prerequisites or return value details, but the output schema likely covers returns.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description fully compensates by explaining each parameter: targets (optional whitelist, default null), stop_on_error (default False with example), delay_ms_between (default 100ms with reasoning). All semantics are clear.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Run the full chain of OpenSIPS data-table reloads' with the verb 'run' and resource 'full chain', and distinguishes from sibling single-module reloads by emphasizing dependency order and batch execution.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the tool's purpose as a full-chain reload and covers parameter guidance (targets whitelist, stop_on_error, delay_ms_between). However, it does not explicitly contrast with alternative single-module reloads (e.g., dispatcher_reload) or state when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description takes full responsibility. It explains the algorithm (username:realm:password and username@domain:realm:password) and return format. It could be more explicit about side effects, but none are expected for a hash generation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured with clear sections for parameters and returns. Every sentence serves a purpose without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (3 params, no nested objects), the description covers purpose, parameters, and return value completely. Output schema existence is acknowledged in context, and the description explains the return dict.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description provides detailed explanations for each parameter: username, domain, password. This adds substantial meaning beyond the schema's names and types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool generates HA1 and HA1B digest authentication hashes, specifying the verb 'generate' and the resource. It distinguishes itself from siblings like security_audit_config by focusing on hash generation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for SIP digest auth but does not explicitly state when to use or provide alternatives. No guidance on when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It states the tool is stateless and returns counters, but lacks details on performance, side effects, or authorization needs. Adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two sentences plus bullet points. Front-loaded key information without unnecessary words. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and an output schema, the description is complete. It explains the tool's purpose and use cases sufficiently for an agent to decide when to invoke it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist in the input schema, so the description cannot add further meaning. The high schema coverage and zero parameters make this dimension perfect.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it returns stateless reply counters per code and totals, with specific use cases. This distinguishes it from other stats tools like get_statistics or homer_stats.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit use cases (auth load, scanner detection, OPTIONS health) but does not mention when not to use it or suggest alternatives, which would strengthen guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the full burden. It discloses the default for include_ha1 and warns about credential material, and specifies the limit default and maximum. This adds behavioral context beyond the schema. However, it doesn't mention idempotency, rate limits, or side effects (though it's a read operation).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise: a one-line summary followed by a structured parameters section. Each parameter is explained in one line with relevant defaults and warnings. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (3 parameters, an output schema exists), the description covers all essential aspects: input parameters, their defaults, a security warning, and the output format (CSV text). It is complete enough for an agent to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description compensates fully by explaining each parameter: domain (optional filter), include_ha1 (default false, security warning), limit (default and max 5000). This adds significant meaning beyond the plain schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Dump subscribers as CSV text,' specifying both the action (dump) and the output format (CSV). This distinguishes it from sibling tools like subscriber_list (likely JSON) and subscriber_bulk_import (import).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides some context (optional domain filter, security warning for include_ha1, limit cap) but does not explicitly explain when to use this tool versus alternatives like subscriber_list or subscriber_get. The usage is implied but not contrasted.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It correctly describes a read-only operation with scope constraint, which is sufficient for a simple getter. No hidden side effects or auth needs disclosed, but likely unnecessary.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: one sentence for purpose and one for parameter. No fluff, front-loaded with key info.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter getter with an output schema, the description sufficiently covers purpose, scope, and parameter format. Minor improvement could mention return value format, but output schema likely handles that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Provides example format ('$avp(my_var)' -> 'my_var') which adds meaning beyond the schema's bare string type. This helps the agent format the parameter correctly.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Get an AVP value' with explicit scope ('global scope only'), which distinguishes it from sibling tools like avp_delete and avp_set that perform other operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit scope limitation (global only, not per-transaction) and notes that per-transaction AVPs are not MI-accessible, guiding when not to use. No explicit alternative mentioned, but implication is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Captures pre/post snapshots for self-documentation, mentions caps on cps/duration, and requires out_dir in a safe allowlist. No annotations, so description carries full burden; states it's 'safe for any non-toy proxy' but doesn't explicitly disclose if it modifies state.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with a summary and parameter list. Every sentence is informative; no wasted words. Concise yet complete.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, parameters, and key behaviors. Hints at output via 'bench artefact JSON' and snapshots. Output schema exists, so return values need not be detailed. Missing some edge cases but sufficiently complete for a load test tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must compensate. It provides detailed parameter documentation: target as host:port, cps capped at 1000, duration_sec capped at 600, out_dir must be in safe allowlist, and label as optional human tag. Adds significant meaning beyond schema types and defaults.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Run a SIPp REGISTER load test against TARGET' with a specific verb and resource, and distinguishes from sibling tools like bench_quick_invite and bench_summary.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides default values and indicates safety for 'any non-toy proxy', implying when to use. However, it lacks explicit guidance on when not to use or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, but description clearly states it does not write to disk, which is a key behavioral constraint. For a tool with no annotations, this disclosure adds value, though additional info on permissions or side effects would be better.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: two sentences plus a parameter list. Every sentence adds value—purpose, behavior, parameter explanations. No wasted words, front-loaded with the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a validation tool with 3 parameters and an output schema (implied but not shown), the description covers the action, parameters, and key behavior. Could mention what happens on validation failure or diff formatting, but given the tool's focused purpose, it is largely complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but description adds rich semantics: existing_cfg as 'current on-disk configuration text', change_description as 'short natural-language description', new_cfg as 'LLM's rewritten configuration text'. This fully compensates for missing schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it validates an edited OpenSIPS config and returns a unified diff. 'Validate' + 'return unified diff' is specific verb+resource, and it distinguishes from siblings like cfg_validate (no diff) and cfg_diff (no validation).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Does not write anything to disk — the caller chooses to persist after reviewing the diff.' This tells the agent this is a preview step before persisting. Could be improved by contrasting with cfg_validate or cfg_diff for even clearer when-not-to-use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses key behaviors: hop chain walking, automatic rewrites, warning reporting, and validation step. Lacks explicit statement about output format or side effects, but with no annotations, this adds significant value beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Efficient two-paragraph structure: purpose + process, then parameter list. No wasted sentences, front-loaded with purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequately describes what the tool does and its output (migrated config, warnings). Given that an output schema exists, the description covers the essential aspects without needing full return details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description fully compensates by explaining each parameter's role (current config text, source/target versions, validation toggle). Adds meaning beyond the schema's type/title only.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Migrate a configuration between OpenSIPS versions', providing a specific verb and resource. Differentiates from sibling tools like cfg_migrate_plan and cfg_diff through the hop chain description.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context on when to use: automatic migration with hop chain traversal and manual fallback for warnings. Mentions optional validation. Could explicitly contrast with cfg_migrate_plan but still effective.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It explains the pattern matching, the confirm parameter's role (dry-run vs actual deletion), and the domain scope. It lacks details on permissions or irreversibility, but the safety mechanism is disclosed, making it fairly transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise: two short paragraphs with no fluff. The first paragraph states the core action, and the second lists parameters in a structured way. It is front-loaded with the purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (3 parameters, output schema exists), the description covers the essentials: purpose, pattern, dry-run, and parameter details. It mentions the companion tool. However, it does not describe the output schema format, but that is handled separately. It is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It fully explains each parameter: domain (target domain), username_prefix (same as generation), confirm (controls dry-run vs deletion). This adds significant meaning beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: remove test subscribers matching a pattern under a domain. It uses a specific verb ('Remove') and resource ('subscribers'), and distinguishes itself from siblings by mentioning it is a companion to gen_test_subscribers.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context: it is for cleaning up test data after generation with gen_test_subscribers. It also explains the dry-run safety mode. However, it does not explicitly list when not to use or mention alternative tools like subscriber_delete, so it lacks explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden. It explicitly states the tool does not dispatch a call, implying non-destructive behavior. It also mentions the return of translated output and metadata, giving sufficient behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise, with a clear one-line purpose, followed by a structured list of parameters and returns. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter tool with an output schema, the description covers purpose, parameters, and returns completely. No additional context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds a Parameters section with explanations for 'dpid' (Dialplan table id) and 'input' (example format), which goes beyond the schema's title and type. This provides useful context, especially with 0% schema description coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool tests a dialplan translation without dispatching a call. It uses the specific verb 'test' and resource 'dialplan translation', distinguishing it from sibling tools like dialplan_add_rule or dialplan_list_rules.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for testing before actual dispatch ('without dispatching a call'). However, it does not explicitly state when to use this tool versus alternatives like actually dispatching or checking rules, so a slight gap exists.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully bears the burden of behavioral disclosure. It accurately describes a read-only query that groups and counts processes, with no hints of side effects. It lists expected output categories, though it does not mention output format or potential errors. Overall, it is transparent about the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences that front-load the main action and then provide valuable examples. Every word earns its place without extraneous details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters, output schema exists), the description is fully complete. It explains the purpose, output types, and real-world usage examples, enabling an agent to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, and the schema coverage is effectively 100%. The description does not need to add parameter information. According to the scoring guidelines, 0 parameters yield a baseline of 4, which is appropriate here.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool groups and counts OpenSIPS child processes by role, listing specific role examples (UDP receivers, TCP workers, etc.). It distinguishes itself from sibling tools like 'perf_hotspots' or 'get_process_list' by focusing on role-based grouping and misconfiguration detection, leaving no ambiguity about its purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for usage by explaining the tool helps spot misconfigurations, with concrete examples (e.g., children=4 on an 8-core machine). It does not explicitly compare to alternatives like 'perf_memory_report' or 'get_process_list', but the context strongly implies when this tool is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. It indicates a read-only listing of active subscriptions, which is non-destructive. It could mention that it is safe and has no side effects, but the current text is sufficient for a simple diagnostic tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two concise sentences. The first sentence defines the purpose, and the second adds a practical use case. No wasted words, and the important information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and an output schema (implied), the description fully explains the tool's function and its diagnostic value. It is complete for a simple listing operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, and the schema coverage is 100%. Per guidelines, a baseline score of 4 is appropriate since the description does not need to add parameter-specific guidance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List active presence subscriptions (the presence hash table)' with a specific verb and resource, and distinguishes from siblings like 'presence_cleanup' and 'pua_list_publications' by focusing on monitoring subscriptions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit context: 'Useful for BLF diagnosis: if a watcher's key phone button doesn't update, this table shows whether OpenSIPS sees the subscription.' It implies when to use but doesn't explicitly mention alternatives or when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden for behavioral disclosure. It discloses that the tool does not actually reload (safe, no side effects), which is critical. It does not detail output format or permissions, but for a simple preview tool, this is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: just two sentences, no fluff. The first sentence states the core functionality, the second gives practical usage context. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters, output exists but described by schema), the description fully covers what an agent needs: purpose, usage context, and safety (no actual reload). No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so no parameter documentation is needed. Schema coverage is 100% and the description does not need to add parameter info. Baseline score 4 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: preview modules that would be reloaded without actually reloading. It uses a specific verb 'Preview' and resource 'modules', and distinguishes itself from sibling reload tools by emphasizing it is a dry run.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use it: 'when a user is about to press the reload everything button and wants to know the blast radius first.' This provides clear context, though it does not explicitly mention when not to use it or alternatives. Still, it is sufficient for the agent to determine appropriate use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, but the description fully describes what the tool returns (reachability, load, state, stats) and its relationship to rtpengine_show. It suggests a read-only operation, though no explicit side-effect disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core purpose, and contains no extraneous information. Every sentence is necessary and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and an output schema, the description provides good context on output and use case, differentiating from a sibling. It could mention that it is a safe read-only tool, but overall sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, and schema coverage is 100%. The description adds significant value by explaining the tool's output and purpose beyond what the empty schema shows.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns reachability and load for every RTPEngine instance, and explicitly differentiates it from rtpengine_show by mentioning it folds in disabled/enabled state and stats.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage context for operators to decide which instance to shed load, but does not explicitly state when not to use or alternative tools beyond mentioning rtpengine_show.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It states the destructive action but lacks details on side effects, permissions, or error behavior. Adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise yet informative: a one-sentence summary, usage context, and parameter explanation in a structured format. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given output schema exists, description is mostly complete. Explains parameter and usage context well. Could mention return values or failure behavior, but overall sufficient for a single-param tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but description adds meaning: 'Exact SIP Call-ID of the media session to terminate', specifying the parameter's purpose and required precision.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Tear down the RTPEngine media session' with a specific verb and resource, and distinguishes from sibling tools like rtpengine_sessions and rtpengine_show.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says when to use ('when a signalling bug leaves a media session stranded') and when not to ('SIP-level dlg_end_dlg alone won't free the RTP ports'), providing clear guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden and discloses multiple behavioral traits: pure Python UDP, clamping of count and timeout, synthetic default from_uri, and the allow_public safety guard. It does not detail error handling but covers key behaviors.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise: a one-line purpose, a line about implementation, a usage sentence, and a parameter table. Every sentence is relevant and front-loaded. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given an output schema exists (not shown), the description does not need to explain return values. It covers the tool's purpose, parameters, and key behaviors. Minor gap: no mention of read-only nature, but it's implicit.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description fully compensates. It explains target format and default port, count clamping (1-20), timeout clamping (0.2-10), the purpose of from_uri, and the safety intention of allow_public. This adds critical meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Send SIP OPTIONS probes') and the resource (target), and distinguishes from sibling tools like sipp_load_test by emphasizing it's a simple UDP implementation without external dependencies.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly mentions a use case ('can the MCP host even reach this next-hop?') and notes it's a pure Python implementation, implying when to use it. However, it does not provide explicit alternatives or when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description carries full burden. It details marking node disabled, polling dialog count, timeout behavior, and clamping of parameters. It doesn't cover re-enabling or failure modes, but sufficiently describes core behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise and well-structured: brief intro, a key behavioral note, then a parameter table. Front-loaded with purpose. No redundant sentences; every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given presence of output schema, description doesn't need to detail return values. It covers the drain process, timeout behavior, and post-condition instructions. Complete for the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage, but description includes a dedicated Parameters section explaining each parameter's role, including clamping ranges for timeout_sec and poll_interval_sec. This adds significant meaning beyond schema fields.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Gracefully drain a cluster node ahead of maintenance' with specific verb (drain) and resource (cluster node). It explains the process (mark disabled, poll dialogs) and distinguishes from sibling tools by focusing on draining vs. other cluster operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit usage context: maintenance prep, expected to stop OpenSIPS only after confirmation, and recommends combining with init-script/systemd. Does not explicitly mention when NOT to use, but the guidance is clear and practical.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses the tool's behavior: it pulls state from specific sources (dlg_list, ul_show_contact, rtpengine_show, nathelper) and returns a unified view with a diagnosis string. It acknowledges limitations ('a real engineer is still the best judge'), adding transparency. No destructive actions are implied.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear purpose statement, numbered list of data sources, and a dedicated parameter section. Every sentence adds value without repetition or unnecessary detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity and the presence of an output schema, the description adequately covers input, data sources, and output type. It explains what the tool returns (unified view + diagnosis) and when to use it, meeting all contextual needs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, but the description adds a detailed meaning for the single parameter 'callid': 'The SIP Call-ID of the problematic dialog.' This provides essential context beyond the schema's title and type.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it correlates dialog, contact, RTPEngine, and NAT state for one call, specifically for one-way audio triage. It distinguishes itself from sibling tools like dlg_list, ul_show_contacts, rtpengine_show, and nathelper_stats by being a composite that consolidates multiple MI calls into one.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly positions the tool as the canonical 'why is this call one-way audio?' triage tool and mentions it consolidates data from four MI calls, indicating when to use it over individual tools. However, it does not explicitly state when not to use it, such as cases not related to NAT or one-way audio.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden. It explains that disabling may cause unreachability after timeout, giving behavioral insight. It does not cover persistence or scope, but is adequate for a simple toggle.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, uses a clear summary followed by a parameters list, and every sentence adds value. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple boolean toggle, the description covers purpose, parameter, and consequences. Output schema exists but is not needed to explain further. Complete for the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 0% schema description coverage, the description includes a 'Parameters' section that fully explains the 'enable' parameter, including default and effect. This adds significant value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool enables or disables NAT pinging for registered contacts, specifying the action and resource. It is distinct from siblings like 'nathelper_stats' and 'sip_options_ping'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use (toggle NAT keepalive) and provides consequences of disabling with a typical timeout. It does not explicitly mention alternatives or when not to use, but the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of disclosing behavior. It notes that out_dir is validated against an allowlist, system directories are rejected, and the directory is created if missing. The overwrite parameter's default behavior is also explained. It doesn't mention auth or failure modes, but covers the key safety aspects for a file write operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concisely structured as a docstring with a clear summary and parameter explanations. Every sentence adds value without redundancy. It is appropriately sized for the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the description is complete for a file-writing tool. It explains the input thoroughly, including validation constraints and default behavior. No additional information seems necessary.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Since schema description coverage is 0%, the description must compensate fully. It explains each parameter: out_dir as an absolute path to the bundle root, files as a mapping of relative paths to content, and overwrite with its default refusal to overwrite existing files. This adds essential meaning beyond the minimal schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool persists a bundle's file map to disk, using the specific verb 'persist' and referencing the related tool 'observability_generate_bundle'. It distinguishes itself by specifying the source of the files map and the validation of the destination directory, which sets it apart from sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context by referencing the source of the files map (from observability_generate_bundle) and the allowlist validation similar to cfg_save_session. However, it does not explicitly state when to use this tool versus alternatives, or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses it samples evenly-spaced points, returns raw time-series requiring delta computation, and has clamping ranges. Could be more explicit about non-destructive nature but adequately transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is structured with overview, use case, alternative, and parameter details. Front-loaded with purpose. Slightly verbose but every sentence adds value; could be tightened slightly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given tool complexity (sampling) and presence of output schema, description is adequate. It explains the raw time-series return, no need to detail output schema structure. Could mention synchrony or side effects, but overall complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description must explain parameters. It does so thoroughly: duration_sec range, sample_count range, groups default prefixes. Adds significant meaning beyond schema types and defaults.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it 'Sample runtime statistics at a fixed cadence over a short window' and provides a specific use case ('is the proxy under pressure right now?'). It distinguishes from sibling tools like perf_hotspots by contrasting with long-term TSDB solutions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly tells when to use (short-term pressure checks) and when not (long-term observation, suggesting alternatives proto_hep or prometheus). Mentions caps (10 min/60 samples) to manage MCP response size.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses key behaviors: import is a write operation, conflict handling, dry run validation without writing, and row caps. Missing details on authentication or error handling for malformed CSV beyond blank/comment lines, but adequate given no annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise and well-structured: a brief header, bulleted CSV format rules, and a clear parameter list. Every sentence adds value, no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers all necessary aspects for an agent: input format, parameters, conflict handling, and limits. Since an output schema exists, return value details are not needed. Complete for the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 0% schema description coverage, the description thoroughly explains each parameter: csv_content (full CSV text), dry_run (default true, parse only), on_conflict (skip/overwrite/error). Adds essential meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Import subscribers from a CSV blob with conflict handling.' It uses a specific verb (import) and resource (subscribers), and distinguishes from sibling tools like subscriber_create (single) and subscriber_bulk_export.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance on CSV format, row limit (5000), conflict handling options, and recommends using dry_run first. However, it does not explicitly contrast with alternative tools for single subscriber operations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It clearly explains the tool's behavior: it resolves flags into modules/globals and does not produce a config. However, it does not explicitly state side effects, safety, or error conditions, though the nature of the operation implies read-only analysis.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise, using a clear structure with a main sentence and a 'Parameters' section. Every sentence adds value without redundancy. It is front-loaded with the essential purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, output schema exists), the description covers purpose, usage, parameter semantics, and relationship to a sibling tool. It is complete enough for an agent to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning beyond the input schema: it specifies that flags are 'WITH_* flag names (case-insensitive)'. The schema only defines an array of strings without such context. This addition is valuable for agent understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool resolves WITH_* flags into modules and globals, specifies what it does NOT do ('Does NOT produce a config'), and distinguishes from sibling 'cfg_build_from_flags'. This provides a specific verb and resource with clear delineation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells when to use this tool ('if I enable these flags, what modules get loaded and which globals get set?') and pairs it with an alternative ('Pairs with cfg_build_from_flags for the full render'). Also notes flags are case-insensitive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that tool does not execute SIPp, which is crucial behavioral info. No annotations exist, so description carries full burden. Lacks specifics on return format (though output schema exists) and authorization needs, but the non-execution disclaimer is clear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise with a clear structure: intro, behavioral note, parameter list using code blocks. No redundant sentences; front-loaded with purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 7 parameters and output schema existence, description covers core behavior and all parameters. Could mention output content more explicitly (scenario XML + command), but output schema likely fills gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but description provides detailed explanations for all 7 parameters: lists enum values for kind, specifies target format host:port for -rsa, and relates standard SIPp knobs. Adds significant meaning beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it produces a SIPp XML scenario and the shell command to run it. Explicitly says it does NOT execute SIPp, differentiating it from sibling tools like sipp_run_scenario or sipp_load_test.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use: to prepare a scenario for the operator to run on their own host. Warns against using MCP process for load traffic, implying alternatives like sipp_run_scenario for execution.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavior: running opensips -C -f by default, unconditional linting, and the validation skipped reason. It also cites the Summit deck claim, adding context beyond mere functionality.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized: a concise purpose statement, a behavioral paragraph, and a clear parameter list. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 4 parameters, no annotations, and an output schema, the description provides complete context: it explains all parameters, default behaviors, and output validation details. It is self-sufficient for correct tool invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 0% schema coverage, the description explains each parameter in detail: scenario lists examples, params describes template parameters, validate and lint state defaults and consequences. This significantly compensates for the sparse schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Generate an OpenSIPS configuration from a predefined scenario template' with a specific verb and resource. It distinguishes this tool from siblings like cfg_generate_iterative and cfg_generate_m4 by focusing on single scenario generation and validation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It calls itself 'the headline tool' and explains the validate-on-render contract, implicitly indicating primary usage. It describes fallback behavior when the binary is missing, though it lacks explicit when-not-to-use guidance or comparison to alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses key behaviors: default dry-run, need for confirm=True to write, and truncate_first deletes existing rows before inserting. Warns about danger of truncate_first. No annotations provided, so description fully covers behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with brief intro followed by parameter list. Every sentence adds value, no fluff. Includes a crucial warning. Front-loaded with key purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers all important aspects: purpose, parameters, default behavior, and output (returns plan when dry-run). Output schema exists, so not describing return values is acceptable. Complete given tool complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 0% schema description coverage, so description must add meaning. It does so by explaining backup_path as path from db_table_backup, confirm as needed for write, and truncate_first as deleting current rows. Adds significant value beyond bare schema names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Restore a table from a JSON backup produced by db_table_backup', specifying verb and resource, and distinguishes from related siblings like db_table_backup and db_table_list_backups.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explains default dry-run behavior and when to use confirm=True and truncate_first=True, including a warning about the danger of truncate_first. Does not explicitly mention when not to use the tool, but context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses key behaviors: it is a light probe hitting a specific endpoint, returns {ok: false} (not raises) when Homer is not configured, and states that the absence of Homer must not break other tools. This provides transparent behavioral expectations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is highly concise: two sentences with no wasted words. The first sentence front-loads the main purpose. Every sentence adds unique value: purpose, technical detail (endpoint), usage guidance, and error behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is complete for a simple health check tool with no parameters and an existing output schema. It mentions the endpoint and the error case. However, it does not explicitly state the success return value (likely {ok: true}), which is a minor gap given that the output schema is said to exist.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so the baseline is 4. The description does not need to add parameter meaning since there are none. The description already implicitly covers the no-parameter nature by describing a self-contained health check.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the purpose: 'Check whether the configured Homer instance is reachable.' It uses a specific verb ('check') and resource ('Homer instance'), and distinguishes itself from siblings by explicitly mentioning 'homer_search_calls_v7' as a related tool that should only be attempted after this health check.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance: 'Useful when deciding whether to even attempt a homer_search_calls_v7 call.' It also explains the special case when Homer is not configured, returning {ok: false} instead of raising an error, which informs the agent when and how to use the tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![opensips-mcp-server MCP server](https://glama.ai/mcp/servers/OpenSIPS/opensips-mcp-server/badges/card.svg)](https://glama.ai/mcp/servers/OpenSIPS/opensips-mcp-server)

Score Badge

Copy to your README.md:

[![opensips-mcp-server MCP server](https://glama.ai/mcp/servers/OpenSIPS/opensips-mcp-server/badges/score.svg)](https://glama.ai/mcp/servers/OpenSIPS/opensips-mcp-server)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/OpenSIPS/opensips-mcp-server'

If you have feedback or need assistance with the MCP directory API, please join our Discord server