FortiManager MCP Server

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v1.9.2
Disambiguation4/5
Most tools have clearly distinct purposes, with detailed descriptions that differentiate similar actions (e.g., add_device vs add_model_device vs add_devices_bulk). However, the presence of multiple assign_* and execute_script_on_* tools, though distinguishable by resource and target, introduces slight ambiguity that could cause misselection in an agent.
Naming Consistency4/5
The majority of tools follow a consistent verb_noun pattern (e.g., create_address_host, delete_firewall_policy, list_devices). Minor inconsistencies exist, such as mixing 'add' and 'create' for device vs. object operations, and outliers like 'health_check' and 'reload_device_list' that deviate from the pattern.
Tool Count2/5
With 103 tools, the server heavily exceeds the typical well-scoped range of 3-15 tools for an MCP server. While the domain is complex, this number is likely overwhelming for an agent to navigate effectively, resulting in a score of 2 for being too many.
Completeness3/5
The tool set covers core FortiManager operations comprehensively, including device lifecycle, policy management, address/service objects, templates, scripts, and tasks. However, notable gaps exist: no create/update for system or template groups, no VPN management, and limited interface manipulation. Given the high tool count, these omissions are significant.
Average 3.9/5 across 97 of 103 tools scored. Lowest: 2.6/5.
See the Tool Scores section below for per-tool breakdowns.
- 6 of 6 community issues answered or closed in the last 6 months
- 67 commits in the last 12 weeks
- Last stable release on August 12, 2026
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI is passing
This repository is licensed under MIT License.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
Add a glama.json file to provide metadata about your server.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully convey behavioral traits. It states only that the tool 'assigns', which implies a create/update operation, but does not disclose idempotency, destructiveness, locking requirements, or whether the assignment replaces existing ones. This is insufficient for safe invocation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with a clear structure: one-line purpose, a bullet-like list of arguments, and a returns line. No extraneous information, though the returns line adds minimal value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 4 parameters, no annotations, and a sparse description, the tool definition lacks completeness. It does not explain error conditions, required object existence (e.g., template_group must exist), or ADOM context. The returns line is vague. An agent would struggle to use this tool reliably without additional documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must add meaning beyond parameter names. It merely lists the parameter names (adom, template_group, device, vdom) without explaining their roles or constraints. The schema already provides these names, so no added value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('assign') and the resource ('template group to a device'), making the core purpose understandable. However, it does not differentiate from sibling tools like assign_package or assign_system_template, which share the same verb-resource pattern.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., assign_package for provisioning templates). There are no prerequisites, context, or exclusions mentioned, leaving the agent without decision support.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description implies a read operation but does not explicitly state that it is read-only, nor does it disclose any side effects, authentication requirements, or rate limits. Since no annotations are provided, the description should carry the full burden, but it adds little beyond the verb.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured with clear Args and Returns sections. It front-loads the main purpose and avoids unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple get operation with an output schema described, the description covers the return structure and error status. However, it lacks details on when errors occur or how to interpret the 'message' field. The absence of usage context and parameter details reduces completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description merely restates the parameter names and types ('adom: ADOM name', 'name: Service object name') without adding any additional context such as allowed values, formats, or examples. With 0% schema coverage, the description should compensate but fails to do so meaningfully.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves detailed information about a service object, using the verb 'get' and specifying the resource 'service object'. While not explicitly differentiating from siblings like 'get_address', the resource name is distinct enough.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as 'list_services' or 'get_service_group'. There are no explicit conditions, prerequisites, or exclusions mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits such as whether existing assignments are overwritten, the impact on device state, or required permissions. The return type 'Assignment result' is vague and lacks detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short and front-loads the purpose. The Args and Returns sections are clear and structured efficiently, with no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no annotations, the description lacks completeness. It does not cover error conditions, side effects, prerequisites, or the format of the return value. The agent has insufficient context to use the tool safely.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description adds value by listing parameters and stating the default value for 'vdom'. However, it does not explain the meaning of each parameter (e.g., what ADOM is) or provide format constraints beyond names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Assign' and resource 'SD-WAN template to a device', distinguishing it from bulk and system template assignments. However, it does not explicitly mention that it assigns to a single device, which is implied by the name and bulk sibling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like assign_sdwan_template_bulk, assign_system_template, or assign_template_group. No prerequisites or preconditions are mentioned, leaving the agent to infer usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool deletes a group, but does not explain whether the action is reversible, what cascading effects occur, or what permissions are required. The description is too minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise, with no unnecessary words. Every sentence adds value, but it could be slightly expanded to include usage context without being verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that no annotations are present and there is an output schema, the description still lacks details about destructive behavior, permissions, and special cases (e.g., what happens if the group is in use). It is insufficient for a delete operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description adds basic meaning to both parameters: 'adom: ADOM name' and 'name: CLI template group name'. This adds value beyond the schema, though it does not provide format or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete a CLI template group', which is a specific verb ('Delete') and resource ('CLI template group'), and it distinguishes itself from sibling tools like create_cli_template_group, get_cli_template_group, and list_cli_template_groups.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, nor does it mention prerequisites, conditions, or limitations. It only lists the arguments.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description only states it returns 'template group details including member templates', but lacks info on read-only nature, idempotency, or access requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is short and front-loaded with purpose. Includes arg/return sections efficiently. No wasted words, but could be slightly more structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool has an output schema, so return values are covered. However, missing context such as when to use versus list_template_groups or how parameters affect results. Adequate but with gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, yet description only lists parameter names (adom, name) without adding meaning, format, or constraints beyond what schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Get details of a template group' with a specific verb and resource. It distinguishes from sibling tools like list_template_groups (list all) and get_template (single template).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives like list_template_groups or get_template. No prerequisites or exclusions mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description must fully disclose behavioral traits. It only states the action without mentioning side effects (e.g., overwriting existing assignments), authorization requirements, whether the operation is synchronous, or error handling. This severely limits an agent's ability to predict the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: one sentence for purpose, followed by a compact parameter list, and a one-line return statement. It is front-loaded and contains no superfluous information, making it easy for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema (context signal), the description only states 'Returns: Assignment result' without specifying the structure (e.g., success flag, task ID, error details). The tool has 4 parameters and no enums, but the description lacks behavioral context (e.g., prerequisites, effects on device state). An agent would need additional information to use this tool reliably in a workflow.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The 'Args' section adds meaningful descriptions beyond the input schema's titles (e.g., 'ADOM name', 'System template name', 'Device name to assign', 'VDOM name (default: root)'). With schema description coverage at 0%, this compensation is effective, though some parameters (like adom) could benefit from more precise context (e.g., which ADOM scope applies).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action: 'Assign a system template to a device.' This specific verb+resource pairing distinguishes it from sibling tools like assign_sdwan_template and assign_system_template_bulk, which handle different template types or bulk operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., assign_system_template_bulk for multiple devices, assign_template_group for group assignments). An agent would have to infer context from the parameter list, which is insufficient for informed selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of behavioral disclosure. It does not mention whether the assignment overwrites or merges, what happens on partial failure, or required permissions. This lack of detail limits the agent's ability to anticipate side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise with four lines of text plus the docstring-style args list. Every sentence is functional, though the format could be more structured (e.g., bullet points).
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is minimal for a bulk operation. Although there is an output schema (not shown), the description only says 'Assignment result' without hinting at the structure or success/error handling. It is complete enough for basic usage but lacks depth.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It lists the three parameters and provides an example format for the 'devices' array, adding some meaning beyond the schema. However, it does not document constraints or allowed values for adom or template.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Assign a system template to multiple devices.' It uses a specific verb 'assign' and resource 'system template', and the 'bulk' suffix implies operating on multiple devices, distinguishing it from similar tools like assign_system_template.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, nor any prerequisites or conditions. For a bulk operation among many similar assignment tools, explicit usage context is missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must convey behavioral traits. It mentions creating an empty template but omits details on permissions, idempotency, error behavior (e.g., duplicate names), or side effects. The description lacks sufficient behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with a clear header, a brief paragraph, and an Args/Returns section. It is well-structured and front-loaded with the main action. No extraneous text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 3 parameters, no annotations, and an output schema, the description covers the basic purpose and parameters but lacks guidance on usage context, prerequisites, and behavioral details. It is minimally adequate but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% (no descriptions in schema), so the description's parameter explanations add value but are minimal: 'ADOM name', 'SD-WAN template name', 'Optional description'. These are basic synonyms of the parameter names. No constraints, defaults, or format hints beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a new SD-WAN template' and explains it creates an empty template for later configuration with interfaces, SLAs, and routing rules. This distinguishes it from sibling tools like assign, delete, get, and list operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide guidance on when to use this tool versus alternatives (e.g., when to create vs clone, prerequisites like existing ADOM, or situations where an empty template is needed). No explicit when-to-use or when-not-to-use information.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behavior. It only states the return includes 'settings and scope,' but does not explicitly confirm idempotency, side effects, or whether authentication is required. The 'get' nature implies read-only, but this is not stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with a single-line purpose followed by labeled Args and Returns sections. Every word adds value, and there is no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that an output schema exists, the description need not detail return values extensively. However, it omits context such as the required ADOM scope and the uniqueness of template names. For a simple retrieval tool, the description is adequate but leaves some gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, yet the description adds no meaning beyond parameter names ('adom' and 'name'). It does not specify formats, valid values, or relationships, forcing the agent to rely solely on parameter names which are moderately self-explanatory but lack depth.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Get details of a specific provisioning template,' which is a specific verb+resource combination. Among sibling tools with get_* prefixes, this one uniquely targets provisioning templates, making its purpose distinct.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like list_templates or other get_* tools. It does not mention conditions, exclusions, or contextual prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears the full burden. It only states the action and basic parameters, but fails to disclose behavioral traits such as whether the assignment overwrites existing configurations, if it is synchronous or asynchronous, whether the ADOM must be locked, or error handling. This is insufficient for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, with a brief summary followed by parameter explanations. It avoids unnecessary words. However, it could be slightly more structured (e.g., using bullet points) for readability, but overall it is efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (writes to multiple devices) and lack of annotations, the description is incomplete. It does not cover prerequisites (e.g., adom lock), side effects (e.g., overwriting existing assignments), error scenarios, or execution outcomes beyond a vague 'Assignment result.' The output schema exists but does not compensate for missing behavioral context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description adds meaningful parameter semantics: 'adom: ADOM name', 'template: SD-WAN template name', and 'devices: List of devices [{"name": "dev1", "vdom": "root"}, ...]' – clarifying the expected structure of devices. This compensates for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Assign an SD-WAN template to multiple devices,' specifying the verb (assign), resource (SD-WAN template), and scope (multiple devices). It distinguishes from sibling tools like assign_sdwan_template (single device) and assign_system_template_bulk (system template).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide guidance on when to use this tool versus alternatives like assign_sdwan_template (single device) or assign_system_template_bulk. It lacks explicit when-to-use, when-not-to-use, or prerequisite information.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description only notes that it deletes a script and returns a result, but without any annotations to supplement, it fails to disclose potential side effects (e.g., irreversibility, dependencies, permission requirements). The behavioral profile is insufficient for an agent to assess risk.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at two sentences plus parameter/return lines. It is front-loaded with the primary action 'Delete a CLI script' and contains no redundant information, effectively communicating the core functionality.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 params, no annotations) and the presence of an output schema (not detailed), the description is minimally adequate. It covers the basic action and parameters but lacks context on return value structure, error conditions, or integration with other tools, leaving gaps for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description must compensate. It identifies 'adom' as an ADOM name and 'name' as the script name, adding minimal meaning beyond the schema titles. However, no additional details like format, constraints, or examples are provided, resulting in limited added value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Delete a CLI script', using a specific verb (delete) and resource (script), which clearly identifies the tool's function. Sibling tools like create_script, update_script, and get_script further distinguish it, confirming its unique purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like delete_device or delete_address. There is no mention of prerequisites, conditions, or when not to use it, leaving the agent without contextual decision support.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states that the tool returns a Task ID for monitoring, implying asynchronous execution, but it does not disclose other behavioral traits such as whether it modifies state, required permissions, error handling, or limits on the number of devices.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short and front-loaded with the purpose. It uses a simple Args/Returns structure without redundancy. However, a bit more detail on parameters would not harm conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that an output schema exists (not shown), the description appropriately mentions the return value (Task ID). However, with three required parameters and no schema descriptions, the description is minimal and could provide more context about parameters and usage flow.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain parameters. It only lists names (adom, script, devices) without any details on formats, constraints, or valid values. For example, it does not specify whether devices are hostnames or serial numbers, or what constitutes a valid script name.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Execute a CLI script on multiple devices,' which is a specific verb-resource combination. It distinguishes from siblings like 'execute_script_on_device' (single device) and 'execute_script_on_device_group' (device group) by the plural 'devices' and the mention of multiple devices in the purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies that this tool is for multiple devices, but it does not explicitly state when to use it versus alternatives like 'execute_script_on_device' for a single device or 'execute_script_on_device_group' for a group. No exclusions or prerequisites are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavioral traits. It implies a read operation ('Get details') and states return content, but fails to mention permissions, side effects, rate limits, or availability constraints. The description lacks sufficient transparency for a tool with no annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is brief and uses a structured docstring format with Args and Returns sections. It avoids unnecessary words, but the Args section could be more concise by integrating into the main sentence. Still, it is well-organized and straightforward.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the basic purpose, parameters, and return value. Given the tool's simplicity (2 params, no nested objects), it is mostly complete. However, it omits potential error conditions, authentication requirements, or relationship to other templates (e.g., system vs CLI templates). With no annotations and no output schema details, a bit more context would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, leaving the description responsible for explaining parameters. It only restates the parameter names ('adom: ADOM name', 'name: System template name') without providing format, valid values, examples, or contextual meaning. This adds minimal value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Get details' and identifies the resource 'system template'. It clearly distinguishes from sibling tools like list_system_templates (which lists all) and assign/unassign system templates (which modify). The description includes the return value scope, making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states the required parameters (adom and name), which indirectly tells when to use the tool. However, it does not explicitly provide guidance on when to use this tool versus alternatives like get_template or list_system_templates. No exclusions or context about prerequisites are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden but only states 'Update a service object.' without disclosing behavior like whether updates are partial/full, idempotency, or side effects. The return format is specified but insufficient for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise, front-loaded with purpose, and structured with Args/Returns sections. Every sentence is informative with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers return values via output schema mention and descriptions, but lacks context on side effects, error cases, or prerequisites. Adequate for a simple update tool but could be more complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description adds value by explaining each parameter's purpose (e.g., 'New TCP port range'), but lacks format details (e.g., '80-443' or comma-separated). Adequate but not comprehensive.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update a service object.' with specific updatable fields (tcp_portrange, udp_portrange, comment), distinguishing it from sibling tools like update_address or delete_service.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., create_service, delete_service) or prerequisites. The description only states what it does, not when to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It states 'create' (write operation) and 'Returns: Created CLI template group', but does not disclose potential side effects, error handling, or whether the operation is idempotent. Limited transparency for a mutable action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is very short and frontloaded with the purpose. The use of 'Args:' and 'Returns:' provides a clean structure. However, it could be slightly more structured (e.g., bullet points) but is already efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Simple creation tool with output schema available, so description covers basic purpose and parameters. However, it lacks context about prerequisites (e.g., does the ADOM need to exist?) and behavior on conflicts. Adequate but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and the description provides basic mappings for each parameter (e.g., 'adom: ADOM name'). This adds some meaning beyond the raw schema, but lacks examples, constraints, or context about valid values, making it minimally helpful.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a new CLI template group' with a specific verb and resource. The name itself distinguishes it from sibling create tools for other entities like packages or scripts, making purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. The name implies its purpose, but there is no mention of prerequisites or exclusions. Usage is implied but not elaborated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavior. It only states it creates a script and returns details, but fails to mention side effects like overwriting existing scripts, required permissions, or idempotency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is structured as a clear list of arguments and a return note. It is concise without unnecessary fluff, though it could be slightly more compact.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the description covers all parameters and mentions return value, it lacks details on error handling, prerequisites, or what the output schema contains. With 6 parameters and an output schema, more context would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description compensates by explaining all parameters, including enum values for script_type and target with their defaults. However, 'adom' is only described as 'ADOM name', lacking deeper context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a new CLI script.' with a specific verb and resource. It distinguishes from siblings like execute_script_on_device and update_script through the action of creation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., execute_script_on_device, update_script). The description lacks context for appropriate use cases or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It only notes the assignment precondition but omits crucial details such as irreversibility, permission requirements, error behavior when the template is still assigned, and side effects on associated configurations. This is insufficient for a deletion tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, using two sentences plus a compact Args/Returns structure. Essential information is front-loaded. No unnecessary words or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the basic purpose and a key precondition, but given the lack of annotations and low parameter coverage, it fails to provide sufficient behavioral context. The presence of an output schema mitigates the need for exhaustive return details, but missing ownership, error cases, and side effects leave gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. However, it merely restates the parameter names ('ADOM name', 'SD-WAN template name') without providing additional context like valid formats, constraints, or examples. The added value over the schema is minimal.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete') and the resource ('SD-WAN template'), making the tool's purpose unambiguous. It is distinct from sibling tools like create_sdwan_template, assign_sdwan_template, or unassign_sdwan_template, so no confusion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes a useful precondition ('Template must not be assigned to any devices before deletion'), but it does not provide explicit guidance on when to use this tool versus alternatives. It lacks instructions on unassigning the template first, which is a prerequisite.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses the return format and error behavior ('Error message if failed'), which is adequate for a read-only operation. However, it does not discuss permissions or potential side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, using a structured docstring format without fluff. It efficiently conveys purpose, parameters, and return structure. No unnecessary sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 required parameters, no nested objects), the description covers purpose, parameters, and return format adequately. An output schema exists but the description still outlines return keys, adding clarity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must compensate. It explains 'adom: ADOM name' and 'name: Address group name', adding essential meaning beyond the empty schema. The explanations are clear, though could be more detailed about 'ADOM'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Get detailed information about an address group,' which is a specific verb+resource. However, it does not distinguish from siblings like 'get_address' or 'list_address_groups', slightly reducing clarity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as 'get_address' (for a single address) or 'list_address_groups'. The description lacks any context on appropriate usage scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the burden. It states the tool returns a log with content and execution time, implying a read-only operation, but it does not explicitly confirm non-destructive behavior or any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with a clear structure: purpose statement, then Args and Returns sections. No extraneous information, though the Args section could be integrated more smoothly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 parameters, 1 required, no nested objects, output schema present), the description covers the essential inputs and output. However, it lacks usage context and behavioral notes, which are important for an agent to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description adds minimal meaning: 'adom: ADOM name' and 'device: Optional device name to filter logs'. This provides basic understanding but lacks format, constraints, or examples, leaving room for ambiguity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool 'Get the latest script execution log', using a specific verb and resource. It distinguishes from sibling tools like get_script_log_output and get_script_log_summary by focusing on the 'latest' log.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide any guidance on when to use this tool versus alternatives. It fails to mention prerequisites, limitations, or context for use, leaving the agent to infer from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and description lacks details on side effects, reversibility, or required permissions. Only states the basic action without behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Very concise with one introductory sentence and structured Args/Returns. No wasted words, though Returns section is minimal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple mutation tool with 4 parameters. Lacks behavioral and usage guidance, and output schema not explained despite its existence.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage, but the Args section adds meaning: 'ADOM name', 'SD-WAN template name', etc. Provides default for vdom. Could be improved with format or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action 'Unassign an SD-WAN template from a device.' It uses a specific verb and resource, and distinguishes from sibling tools like assign_sdwan_template.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., unassign_system_template). No prerequisites or context provided beyond the basic action.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description reveals that 'members' replaces the existing list and the return format includes status/message. However, it does not clarify if other fields are additive or replaced, nor does it mention side effects or validation behavior. With no annotations, more detail is expected.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with a clear Args/Returns structure. Every sentence adds information, though the formatting could be tighter. It is front-loaded with the action and fits in a few lines.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple with four parameters and an output schema. The description covers action, parameters, and return value but omits whether updates are partial or full replacement (except for members). Prerequisites and error conditions are not discussed, leaving some gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It adds value for 'members' (replaces existing) and 'comment' (new comment), but 'adom' and 'name' are just restated. The descriptions are minimal but partially informative.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update an address group.' with input parameters that match the resource. Siblings include create, delete, get, and list for address groups, making the purpose distinct and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like create_address_group or delete_address_group. It lacks context on prerequisites (e.g., group must exist) or conditions for updates.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses the failure condition (address in use) and return structure, but lacks details on permissions, side effects, or behavior when the address does not exist. Adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (2 sentences plus structured Args/Returns) and front-loads the purpose and warning. Every sentence serves a purpose, though the Args/Returns could be more concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a simple delete tool with an output schema, the description covers the purpose, a key failure condition, and return format. Missing details like behavior for non-existent addresses, but overall sufficient for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must explain parameters. It provides basic explanations ('ADOM name', 'Address object name to delete'), which adds minimal value over the raw schema. With only two simple parameters, this is acceptable but not exemplary.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Delete' and the resource 'firewall address object'. Among sibling tools like 'delete_address_group' and 'delete_firewall_policy', this tool's purpose is well differentiated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes a warning about failure when the address is in use, but does not provide explicit guidance on when to use this tool versus alternatives like 'update_address' or other delete tools. No alternatives are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states the tool deletes (mutation) and may fail, but it does not disclose permission requirements, reversibility, or side effects (e.g., no mention of cascading effects). The return structure is described, but behavioral traits are minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: one sentence for purpose, one warning, two parameter clarifications, and a return type. Every sentence earns its place with no redundancy. The structure front-loads the action and warning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity (two required string parameters, no nested objects), the description covers the basic operation and output schema. However, it omits error cases beyond the warning, whether the tool is idempotent, and the expected behavior if the group does not exist. For a mutation tool, this is adequate but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides only titles ('Adom', 'Name'), but the description adds meaning by labeling them 'ADOM name' and 'Group name to delete', which clarifies the purpose of each parameter. With 0% schema description coverage, this compensation is effective, though it could be more detailed (e.g., allowed values).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete an address group.' This is a specific verb and resource combination. Among siblings like delete_address, delete_service_group, etc., it is distinct because it explicitly targets address groups, not individual addresses or other objects.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes a warning that deletion will fail if the group is in use by policies, but it does not provide explicit guidance on when to use this tool versus alternatives (e.g., update_address_group or delete_address). There is no mention of prerequisites or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description only mentions a return value (Task ID). It does not disclose behavioral traits such as whether the operation is destructive, requires special permissions, or has side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at four short lines, front-loading the action and then listing args and returns. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the description covers parameters and return value, it omits context about execution behavior (e.g., immediate vs. queued, error handling). The presence of an output schema reduces the burden, but gaps remain for a script execution tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage (only titles like 'Adom'), but the tool description adds meaning by explaining each parameter (e.g., 'adom: ADOM name'). This provides necessary semantics beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Execute a CLI script on a device group,' using a specific verb and resource. This distinguishes it from siblings like execute_script_on_device and execute_script_on_devices, which target different scopes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., execute_script_on_device for single devices). The description lacks context about appropriate scenarios or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It only states 'Get details' but does not specify if it is read-only, authentication requirements, error behavior, or any side effects. This is insufficient for a tool with no annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with three lines, front-loaded with the main purpose, and includes a clear Args/Returns structure. No extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the low complexity (2 required simple string params) and presence of output schema, the description covers purpose and parameters adequately. However, it lacks usage guidelines and behavioral transparency, leaving some gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, but the description adds 'ADOM name' and 'SD-WAN template name' for the two parameters. This provides basic meaning beyond the schema titles but lacks format, constraints, or examples.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Get details of a specific SD-WAN template,' providing a specific verb and resource. It distinguishes from siblings like 'list_sdwan_templates' which lists all templates, and other get_* tools for different resources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the name and description (use when you need details of a specific template), but there is no explicit guidance on when to use this tool vs alternatives like 'list_sdwan_templates' or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description only mentions the return structure but fails to disclose behavioral traits such as authentication requirements, rate limits, or whether it is a read-only operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with sections for args, returns, and an example. It is appropriately sized for the tool's simplicity, though could be slightly more concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the low complexity and the presence of an output schema (described), the description covers the return values adequately. It lacks pagination or limit details but is generally complete for a list operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description explains the 'fields' parameter meaningfully: 'Specific fields to return (optional, returns all if not specified)'. This adds value beyond the schema, which has 0% description coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists all Administrative Domains (ADOMs) in FortiManager, using a specific verb and resource. It distinguishes from sibling 'get_adom' implicitly by being a list operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives like get_adom. It does not mention when not to use it or any prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It explains the default ADOM behavior and return fields (name, type, assigned devices), but does not mention idempotency, pagination behavior beyond the limit parameter, or any rate limits/authentication needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, front-loaded, and every sentence provides value. It efficiently covers purpose, parameter details, and return values without fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (list with 2 optional params, output schema exists), the description covers purpose, parameter semantics, and return fields adequately. It could mention if there is any filtering beyond ADOM, but it is sufficiently complete for a list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description compensates by explaining the adom parameter's default derivation and the limit as maximum number. This adds meaning beyond the bare schema properties.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists system templates (device profiles) in an ADOM, using a specific verb and resource. However, it does not differentiate from sibling tools like list_templates or list_sdwan_templates, which could be ambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives is provided. The description lacks any 'when to use' or 'when not to use' context, leaving the agent to infer usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behaviors. It only states the action and returns 'Unassignment result' without mentioning permissions, reversibility, or potential side effects. This is minimal and insufficient for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with a clear structure: one-line summary, Args list, Returns. No redundant information; every sentence is relevant and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists, return value details are not needed. However, the description lacks context about prerequisites (e.g., template must be assigned), error handling, or typical usage scenarios. It is adequate but has clear gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, but the description adds meaning by labeling each parameter (e.g., 'ADOM name', 'Device name') and noting the default for vdom. This clarifies purpose beyond bare property names, though it lacks constraints or formatting details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with a specific verb ('Unassign') and resource ('system template' from a device). It directly distinguishes from sibling tools like assign_system_template.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., assign_system_template, assign_system_template_bulk). No context on prerequisites or expected scenarios is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description only says 'Update an existing CLI script,' indicating mutation but lacks details on side effects, error conditions, or idempotency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, using a clear docstring format with sections for Args and Returns. No unnecessary information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description explains the basic purpose and parameters but does not address partial updates, validation rules, or output format details. Adequate but not comprehensive for a 6-parameter mutation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema coverage, the description adds value by listing each parameter with a brief explanation (e.g., 'adom: ADOM name', 'content: New script content'), clarifying their role beyond the schema's property names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Update an existing CLI script,' clearly indicating the verb and resource. It distinguishes from sibling tools like create_script and delete_script.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives, prerequisites (e.g., script must exist), or conditions that would make another tool more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It states it 'gets' information, implying a read operation, but does not explicitly confirm read-only status, side effects, or authentication requirements. The return format is described, but overall behavioral context is lacking.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, uses a clear docstring format with Args and Returns sections, and front-loads the purpose. Every sentence adds value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (2 params, read operation), the description covers purpose, arguments, and return values. The presence of an output schema reduces the need for detailed return documentation. However, it could mention any prerequisites or access requirements for completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has two required parameters with only titles. The description adds 'ADOM name' and 'Address object name', providing slight clarification over the schema. However, schema coverage is 0% according to context, so the description partially compensates but does not fully enrich parameter meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with 'Get detailed information about a firewall address object,' using a specific verb and resource. This clearly distinguishes it from sibling tools like list_addresses (which lists all addresses) and other get_* tools for different objects.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implicitly suggests using this tool when detailed information is needed rather than just a list, but it does not explicitly state when to use it versus alternatives. No exclusions or alternative recommendations are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description carries the burden. It states that the tool returns script details including content, which is useful. However, it does not mention any permissions, rate limits, or side effects. Since it is a read operation, the description is minimally adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, with three short sections (purpose, args, returns). Every sentence serves a function, and the purpose is front-loaded. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (not shown but confirmed), the description need not detail return values but still mentions 'details including content'. For a simple getter with two required parameters, the description is reasonably complete. It lacks details on permissions or error conditions, but that is acceptable for a read tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has two parameters (adom, name) with no descriptions. The description adds 'ADOM name' and 'Script name', which only restates the property names without adding significant semantic detail. With schema coverage at 0%, the baseline is 3, and the description barely compensates.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Get details of a specific CLI script.' This uses a specific verb (get) and resource (CLI script), distinguishing it from sibling tools like create_script, delete_script, and get_script_log_latest which deal with different aspects.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention that list_scripts should be used to enumerate scripts, or that get_script_log_* tools are for logs. No when-not-to-use or exclusion criteria are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full burden. It only states 'Check health and connection status' without disclosing what endpoints are tested, how health is determined, or any behavioral details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single efficient sentence with no wasted words. However, it could benefit from slightly more detail without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has zero parameters and an output schema exists (not shown), the description is adequate but lacks context about what 'healthy' means or any side effects. It is minimally complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so the schema is trivially covered. The description adds no parameter-specific meaning, but for zero-parameter tools the baseline is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Check') and the resource ('FortiManager MCP server health and connection status'). It distinguishes this tool from sibling tools that perform CRUD or management operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage as a preliminary health check but provides no explicit when-to-use or when-not-to-use guidance, nor does it mention alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are absent, so description carries full burden. It implies a read-only listing operation, which is low-risk, but lacks details on any behavioral constraints like pagination limits or permission requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description includes an extra sentence explaining what CLI template groups are, which is background context. While not extremely verbose, it could be streamlined. The Args/Returns structure is clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple nature of the tool and the presence of an output schema (not shown), the description covers the purpose, parameters, and return type adequately. No major gaps identified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage, and the description compensates by explaining both parameters: adom (with default logic) and limit (maximum number). This adds meaningful context beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'List' and the resource 'CLI template groups' scoped to an ADOM. It is distinct from siblings like 'get_cli_template_group' (singular) and 'list_template_groups' (without CLI prefix), though not explicitly differentiating.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides basic understanding of the tool's function but offers no guidance on when to use this tool over alternatives or any prerequisites. No mention of when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden of behavioral transparency. It states 'Creates a copy' indicating a write operation, but it does not disclose important behaviors such as whether existing packages with the same name are overwritten, if the operation is idempotent, or any required permissions. The return format is described, but side effects are understated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is reasonably concise and well-structured with a clear docstring format (Args, Returns, Example). It includes only necessary information without redundancy. However, the example is a bit lengthy but adds value, so not penalized heavily.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and the presence of an output schema in the description, the context is fairly complete. It covers the function, all parameters, return structure, and an example. It could improve by mentioning preconditions (e.g., the source package must exist) or error behavior, but overall it is adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, so the description's parameter details ('adom: ADOM name', 'package: Source package name', 'new_name: Name for the cloned package') add essential meaning beyond the schema. While concise, these descriptions compensate for the lack of schema annotations, meeting the baseline expectation for 0% coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Clone a policy package. Creates a copy of an existing package with all its policies.' It uses a specific verb ('clone') and resource ('policy package'), and it distinguishes from siblings like create_package or delete_package by specifying that it duplicates an existing package.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use this tool (when you want to duplicate a package), but it does not explicitly state when not to use it or mention alternatives such as create_package. No guidance on prerequisites or comparisons is provided, so usage guidance is only implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It does not mention idempotency, side effects, authentication requirements, or what happens on duplicate IPs. The return format is described, but safety and mutability traits are missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is structured with Args, Returns, and Example sections. It is longer than minimal but every part adds value: purpose, parameter explanations, return format, and a concrete example. No redundant sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 4 parameters (3 required) and an output schema, the description covers parameters, provides an example, and describes the return value. It is complete for a creation tool, though behavioral context is missing. The presence of an output schema in context reduces the need to explain return values, but the description does it anyway.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description compensates by explaining each parameter: adom (ADOM name), name (object name), ip (with example), comment (optional). The return schema is also described in text. This adds significant meaning beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Create a host address object (single IP)', providing a specific verb and resource. The tool name includes 'address_host', and the description clarifies it's for single IP addresses, distinguishing it from siblings like create_address_fqdn or create_address_range.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly mention when to use this tool over alternatives. While the purpose 'single IP' implies the use case, there is no guidance on when not to use it or comparison with other address creation tools. The example provides context but not explicit usage rules.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses that a task is created, a task ID is returned, and the session must be kept alive. Missing details about failure handling, permissions, or side effects, but provides adequate transparency for a write operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (4 sentences plus Args/Returns) with a clear structure: main action, requirement, behavioral note, important warning, and parameter list. No unnecessary words, but the Args section is as brief as the schema titles.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (script execution), the description covers the basic workflow but lacks details on prerequisites, error handling, or script availability. Output schema is not provided, but the return value (Task ID) is mentioned. More context about what ADOM is would help.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% (no parameter descriptions in schema), so the description must compensate. It lists the three parameters (adom, script, device) with brief one-word descriptions (e.g., 'ADOM name'), but does not explain format, expectations, or how to find the script name. This adds minimal value beyond the parameter names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Execute') and resource ('CLI script on a specific device'), and differentiates from sibling tools like execute_script_on_device_group by specifying a single device target.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context: the script must have target=remote_device, and execution creates a task that should be monitored. However, it does not explicitly state when not to use this tool or compare it with siblings like execute_script_on_devices.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Basic behavioral info: associates package with devices for later installation. Does not disclose side effects, permissions, or safety profile. No annotations exist, so description carries the burden but provides limited detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise and well-structured: brief statement, then Args, Returns, and Example. No redundant information, every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, parameters, return values, and example. However, lacks details on prerequisites (e.g., existence of ADOM/package/devices) and does not clarify synchronous/asynchronous behavior. Output schema is described in Returns.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
While schema coverage is 0%, the description adds meaning by listing and describing each parameter (adom, package, devices with format example), compensating for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Assign a policy package to devices.' with specific verb and resource, distinguishing it from sibling tools like assign_sdwan_template or assign_system_template.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. It lacks context on prerequisites, when not to use, or comparison with similar assign tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It only states the creation action and return format, without addressing potential side effects, permissions, or constraints (e.g., handling duplicates, overwrite behavior). This is minimal transparency for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured as a docstring with clear sections (Args, Returns, Example). It is comprehensive without being verbose, though the opening line could be slightly more concise. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (4 parameters, output schema defined), the description provides a complete picture: what it does, required inputs, acceptable formats, and expected return values. It lacks edge-case behavior (e.g., duplicate handling) but is sufficient for standard use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% coverage (no descriptions), but the description compensates fully by explaining each parameter: adom, name, subnet (with example formats), and comment (optional). This adds significant meaning beyond the schema's bare titles.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Create a subnet/network address object,' using a specific verb+resource. Among sibling tools like create_address_fqdn, create_address_host, etc., it clearly distinguishes this tool as specializing in subnet addresses.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for creating subnet addresses by specifying the subnet parameter format (CIDR or netmask). However, it does not provide explicit guidance on when to use this tool versus alternatives (e.g., for FQDN or host addresses), nor does it mention when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It describes a creation operation and return format, but fails to mention error handling (e.g., duplicate group name), required permissions, or whether the group can be updated later. Missing typical concerns for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: one-line summary, bullet-style Args, Returns, and an example. It is front-loaded with the core purpose, and every sentence is informative. No redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers purpose, parameters, return format with keys, and a concrete example. It is fairly complete for a simple create tool, but lacks mention of prerequisites (e.g., whether member services must exist) and conflict behavior. Still, it provides a solid functional contract.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It adds meaning by defining each parameter (e.g., 'members: List of service names to include') and clarifying the optional comment. The example further illustrates usage. This exceeds baseline, though param descriptions are brief.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a service group' with a specific verb and resource. It distinguishes from sibling tools like create_service_icmp and create_service_tcp_udp by focusing on grouping existing services, making its purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool versus alternatives like create_service_icmp or list_service_groups. While the function signature implies grouping, there is no guidance on selection criteria or when not to use it (e.g., for single-service creation).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It explicitly warns that the package and all its policies are deleted and that the operation cannot be undone, covering the key behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded with the purpose and warning. The Args/Returns section is clear but adds verbosity; could be more compact.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with two parameters, the description covers inputs, warnings, and return format. It lacks prerequisites like ADOM lock status, but overall is adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It adds only basic labels ('ADOM name', 'Package name to delete') without further context like format, allowed values, or examples.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the verb ('Delete') and resource ('policy package'), and the warning distinguishes it from non-destructive sibling tools like `get_package` or `clone_package`.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes a warning about irreversibility but does not explicitly state when to use this tool versus alternatives like `delete_firewall_policy` or `update_package`. Usage context is implied but not directive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the failure condition (group in use) and the return format (status/message dictionary), but lacks details on permissions, irreversibility, or side effects beyond the warning.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: a one-liner purpose, a critical warning, then clear Args and Returns sections. Every sentence serves a purpose without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the low complexity (2 simple parameters) and the presence of an output schema (emulated via the Returns block), the description covers all essential aspects: purpose, parameter semantics, failure condition, and return format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description adds meaning by explaining 'adom: ADOM name' and 'name: Group name to delete,' which clarifies the parameter roles beyond the schema's generic titles.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete a service group,' specifying the verb and resource. However, it does not distinguish from sibling tools like delete_service, which deletes individual service definitions, leaving ambiguity about when to use this tool versus alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes a warning that deletion will fail if the group is in use by policies, which provides important context for when not to use the tool. However, it does not explicitly define when to use this tool or mention alternative tools for related operations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description fails to disclose side effects, required permissions, or potential impacts like overwriting existing settings or service restarts. Return dict partially covers outcome but not behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with main purpose, includes parameter list and return structure. Not overly verbose but could be slightly more concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Provides basic understanding of purpose, parameters, and return values. Lacks preconditions, side effects, or installation behavior details, which would be expected for a device modification tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds meaning to both parameters: adom as ADOM name and devices as list of objects with name and vdom. Clarifies structure beyond bare schema, though schema has 0% description coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool installs device settings (interfaces, DNS, NTP) rather than a full policy package, distinguishing it from sibling tools like install_package.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly mentions it installs 'only' device settings 'without policy package', providing clear context for when to use. Lacks explicit 'when not to' or alternatives, but context is strong.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden. It details the return format but lacks information on pagination, rate limits, or any side effects. It is adequate for a simple list operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is structured with clear args and returns sections. It front-loads the purpose. The sentence about address groups is background but not essential; overall it is efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the description covers parameters and output adequately for a simple list, it omits details about environment variable fallback behavior, authentication, and whether the list is paginated. It is acceptable but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description adds meaning by explaining 'adom' uses an env var default and 'name_filter' performs partial matching. This compensates well for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List firewall address groups in an ADOM', which is a specific verb and resource. It distinguishes from siblings like 'get_address_group' (single) and 'list_addresses' (address objects).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for listing multiple address groups but does not explicitly state when to use this tool versus alternatives like 'get_address_group' or mention any prerequisites or when-not-to-use scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It explains the default ADOM from env var and the limit parameter, but does not disclose potential side effects, errors, or that it is a read-only operation. Provides return format but not behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, front-loaded with the core purpose, and structured with Args/Returns. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the existence of an output schema, the description provides necessary context about the return fields. It covers the core operation, parameters, and return format. Missing details on error handling or differentiation from other list tools, but sufficient for a straightforward list operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description adds meaningful semantics for both parameters: 'adom' with default source, 'limit' with purpose. This compensates for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List SD-WAN templates in an ADOM' with a specific verb and resource, and context about what SD-WAN templates are. This distinguishes it from siblings like get_sdwan_template, create_sdwan_template, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives like list_templates or get_sdwan_template. The description only states what it does without suggesting when it is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description describes the return structure (dict with status, count, groups, message) but does not disclose potential side effects, performance characteristics, or whether pagination is supported. The disclosure of defaults and partial match behavior adds some value, but overall transparency is adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with a clear one-line summary, followed by structured Args and Returns sections. While slightly verbose, every sentence serves a purpose. The repetition of parameter names from schema is acceptable given the added semantics.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's two optional parameters and the presence of a return schema, the description covers the core functionality and parameter behavior. However, it lacks guidance on when to prefer this list operation over the 'get_service_group' tool, and does not address potential edge cases like empty results or pagination.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage. The description adds meaningful detail: adom's default value (from env var or 'root') and name_filter's partial match behavior. This compensates for the schema's lack of documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list' and resource 'service groups' scoped to 'an ADOM', distinguishing it from sibling tools like 'get_service_group' (single retrieval) and 'list_services' (different resource).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives such as 'get_service_group' or 'list_services'. Usage is implied by the description but lacks context for decision-making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It states partial update behavior and provides return values, but lacks disclosure of side effects (e.g., immediate traffic impact, need for commit, reversibility, dependency on existing objects). Examples help but do not fully cover behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: opening sentence, parameter list, return values, and examples. It is front-loaded with key behavior. Could be slightly more concise by removing redundant 'New' prefixes, but overall appropriate for the complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (17 parameters, mutation, no annotations, output schema exists), the description covers purpose, partial update, parameter semantics, and return format. However, it lacks context on error conditions, prerequisites (policy must exist), parameter interactions, and scope (adom, package). Examples partially compensate but leave gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description includes a list of parameters with brief explanations (e.g., 'New action - "accept" or "deny" (optional)'), adding meaning beyond the schema's parameter names and types. While some explanations are minimal (e.g., 'New log mode'), overall it provides adequate semantics for a tool with 17 parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it updates an existing firewall policy with partial update semantics ('Only the specified fields will be updated'). The verb 'update' and resource 'firewall policy' are specific, and it distinguishes from sibling tools like create, delete, list, move, and search.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool vs. alternatives, nor does it mention prerequisites (e.g., policy must exist). However, the name and context make it clear it's for updating existing policies, and examples provide implicit guidance. Lacks explicit 'when-not' or alternative suggestions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden for behavioral disclosure. It states that the tool modifies the database (destructive effect) and returns a Task ID for monitoring. It does not mention permissions, error handling, or reversibility, but the disclosed behavior is adequate for a straightforward execution tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise—only four sentences plus the parameter list. It front-loads the core action and constraint, with no redundant or irrelevant information. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (3 parameters, no enums or nested objects, output schema exists), the description covers the main purpose, constraint, and return value. However, it lacks details about potential failure modes, prerequisites, or explicit async polling behavior, which would enhance completeness for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage for parameters, so the description must compensate. It lists the three parameters with brief explanations ('adom: ADOM name', etc.), which adds meaning beyond the bare names in the schema. However, it lacks format constraints, examples, or additional details that would fully compensate for the schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Execute' and clearly identifies the resource 'CLI script against a policy package or ADOM database.' It distinguishes from sibling tools by specifying the target environment (package/ADOM DB) rather than live devices, which other execute_script siblings handle.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit context: the script must have 'target=adom_database' and that execution modifies the package/ADOM DB, not a live device. This helps the agent decide when to use this tool versus live-device alternatives. It does not explicitly list when not to use or alternative tool names, but the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It describes the return format and mentions error messages, but does not explicitly state that the tool is read-only, idempotent, or any authentication requirements. It is moderately transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with clear sections: description, Args, Returns, Example. It is concise with no unnecessary words, and the purpose is front-loaded in the first sentence.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (as noted in context), the description covers return values and error handling. It includes an example. However, it does not mention behavior when the policy does not exist, which is a minor gap for completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds meaning by listing parameters with brief descriptions in the Args section. It also provides an example with concrete values, making the parameter usage clear.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Get detailed information about a specific firewall policy.' It clearly identifies the verb (get), resource (firewall policy), and specificity (by adom, package, policyid). This distinguishes it from sibling tools like 'list_firewall_policies' and 'search_firewall_policies'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'list_firewall_policies' or 'search_firewall_policies'. It does not specify prerequisites, context, or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that the tool returns HA status, cluster members, sync status, and roles, and includes error handling. However, with no annotations provided, it does not explicitly state that the operation is read-only and non-destructive, though this is implied. The description could be more explicit about side effects (none).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear first-line purpose, bullet points for key outputs, a return type explanation, and an example. It is slightly verbose but each section adds value. No wasted sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters, no output schema shown but exists), the description covers return values, error handling, and example usage. It is complete enough for an agent to understand input, output, and behavior. The output schema likely covers the struct, so the description's extra details are additive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0 parameters, so the description does not need to explain parameter meanings. Baseline score of 4 applies. The description correctly avoids adding redundant parameter information.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves FortiManager HA status, using a specific verb ('Get') and resource ('HA status'). It distinguishes from siblings like 'get_system_status' by specifying HA-specific fields such as cluster members and sync status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for checking HA cluster status but provides no explicit guidance on when to use this tool versus alternatives like 'health_check' or 'get_system_status'. No exclusions or prerequisites are mentioned, leaving the agent to infer context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden. It describes the operation as retrieving output, which is non-destructive. It does not mention authentication, rate limits, or error conditions, but the behavior is straightforward and adequately disclosed for a read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured, starting with a clear purpose followed by an Args/Returns section. Every sentence adds value, and there is no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that an output schema exists (per context signals), the description does not need to detail return values. It covers parameter semantics thoroughly and includes helpful derivation logic for log_id. It could mention potential errors or prerequisites, but is otherwise complete for a simple retrieval tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description compensates fully. It explains the purpose of each parameter: adom (ADOM name), log_id (derivation from task_id), and device (optional but required for device-specific logs). This adds significant meaning beyond the input schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Get specific script execution output by log ID', which is a clear verb+resource+method. It implies differentiation from sibling tools like get_script_log_latest and get_script_log_summary through the focus on a specific log ID, but does not explicitly distinguish them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides context for when to use the tool (to get output by log ID) and includes derivation of log_id from task_id. However, it does not explicitly state when not to use it or mention alternatives like get_script_log_latest or get_script_log_summary.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description discloses the return format (list of scripts with fields) and default ADOM behavior. However, it does not mention permissions, rate limits, or edge cases (e.g., empty results). Annotations would have raised the baseline.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is brief (5 sentences) with a clear structure: one-line purpose, Args list, Returns summary. No unnecessary words. Front-loaded with the action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool and the presence of an output schema, the description covers purpose, parameters, and return structure. It lacks error handling info but is sufficient for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description fully documents all 4 parameters: default ADOM from env var, enumerated script_type values, enumerated target values, and limit meaning. This adds substantial meaning beyond the schema titles.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List CLI scripts in an ADOM', using a specific verb and resource. It distinguishes from other listing tools (e.g., list_addresses, list_devices) and is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance is provided on when to use this tool vs alternatives (e.g., get_script for a single script, or search_objects). No when-not-to-use or prerequisites are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description indicates that this tool modifies device metadata but does not clarify whether updates are partial (only provided fields) or full replacements. It also omits any discussion of side effects, permissions, or idempotency. With no annotations, the description carries full burden but only meets a basic level.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured with a one-line summary, separate sections for Args, Returns, and an Example. Every part adds value, and the example illustrates typical usage without excess.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple update tool with an output schema described, the description covers purpose, all parameters, return structure, and usage example. Missing are error conditions, partial vs full update behavior, and prerequisites (e.g., device existence). Still, it is quite complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description provides meaningful explanations for all five parameters in the Args block (e.g., 'adom: ADOM name', 'device: Device name', 'description: New device description', 'latitude: GPS latitude for device location', 'longitude: GPS longitude for device location'), fully compensating for the 0% schema description coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update device properties' and elaborates with 'Modify device metadata like description and location.' This specific verb+resource combination distinguishes it from siblings such as add_device, delete_device, and get_device.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like update_address or other update tools. There is no mention of prerequisites, conditions, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It explains return values and background task behavior via flags, but does not disclose prerequisites, authentication needs, or potential side effects beyond what's implied.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is well-structured with Args, Returns, and Example sections. The main purpose is front-loaded. Slightly verbose but remains clear and organized.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given complexity (3 params, one nested object) and presence of output schema (return value described), the description is fairly complete. Could mention prerequisites or error conditions not covered in the example.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but description provides detailed explanations for all three parameters, including a breakdown of the device object fields. This adds significant meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Add multiple devices to FortiManager in bulk' with specific verb and resource. The 'bulk' qualifier distinguishes it from the sibling tool 'add_device' which adds a single device.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies usage for efficiency when adding many devices ('Registers multiple devices at once for efficiency'), but does not explicitly state when to use instead of 'add_device' or 'add_model_device', nor provides exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden for behavioral disclosure. It states that commits persist pending changes and is required before unlock, which is essential for a commit tool. However, it does not mention side effects, permissions, idempotency, or error handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, with a single-sentence functional description, a critical usage note, and clearly separated args/returns sections. Every sentence adds value with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple commit tool with one parameter and an output schema, the description covers the essential purpose, usage timing, and return format. It could mention error conditions or prerequisites, but is largely complete given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 0%, so the description must compensate. It describes the single parameter 'adom: ADOM name to commit', which adds minimal context beyond the schema property name and type. It does not provide format, constraints, or examples.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Commit changes to an ADOM' using a specific verb and resource, and explicitly mentions 'Saves all pending changes made to the ADOM.' It also distinguishes from sibling tools like lock_adom and unlock_adom by specifying the required order of operations ('Must be called before unlocking to persist changes').
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear guidance on when to use the tool: 'Must be called before unlocking to persist changes.' It implies usage after making changes and before unlocking, but does not explicitly exclude other scenarios or mention alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description warns that the operation cannot be undone and specifies the return format, but lacks details on idempotency, error behavior for non-existent policy, or dependencies. With no annotations, this is adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficient: one sentence for the action, a warning line, and structured Args/Returns sections. No extraneous text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple, and the description covers purpose, irreversibility, parameters, and return structure. It could mention the need for ADOM write access or lock, but overall it is sufficient for a straightforward delete operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds concise descriptions for each parameter (e.g., 'Policy ID to delete') which is valuable given 0% schema description coverage. It clarifies the meaning of each required parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete') and the resource ('a firewall policy'), distinguishing it from sibling delete tools like delete_address or delete_device.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for deleting a single policy by ID but does not provide explicit guidance on when to use this tool versus alternatives like delete_firewall_policies_bulk, nor does it mention prerequisites (e.g., ADOM lock).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of disclosure. It warns that deletion will fail if the service is in use by policies, a key behavioral trait. However, it does not mention that deletion is irreversible or any authentication requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured, with a clear purpose statement, a warning, a list of arguments, and a return value specification. Every sentence provides value without unnecessary verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a deletion tool with 2 parameters and an output schema (implied by the return section), the description covers the key warning and return structure. It is mostly complete, though it could benefit from mentioning permanence of the deletion.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage. The description lists the parameters 'adom: ADOM name' and 'name: Service name to delete', adding basic clarification beyond the bare property names. This is helpful but minimal, only partially compensating for the lack of schema documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete a service object,' which uses a specific verb and resource. This distinguishes it from sibling tools like delete_address, delete_device, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes a warning about failure if the service is in use by policies, which provides some usage context. However, it does not explicitly state when to use this tool vs alternatives like delete_service_group, nor does it offer guidance on prerequisites or conditions for successful use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It correctly indicates a read operation ('Get'), but does not disclose potential side effects, permissions, or error handling beyond 'message' key. Adequate but not thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with Args, Returns, and Example sections. Every sentence is informative, no wasted words. Front-loaded with clear purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given output schema exists, description appropriately focuses on usage. Covers parameters sufficiently, includes example for clarity. Could mention that it requires device name, but that is implicit from required parameter.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 0% schema description coverage, the tool description provides clear explanations for all three parameters: name, adom (with default behavior from env var), and include_details (sub-objects like VDOMs). Adds significant value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Get detailed information about a specific managed device', using a specific verb and resource. It distinguishes from sibling tools like list_devices and get_device_interfaces by focusing on a single device with full details.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage for retrieving full device details, but no explicit guidance on when to use this vs alternatives like list_devices or get_device_interfaces. Does not state when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description implies read-only via 'list' verb and describes return format, but does not explicitly state lack of side effects or error details beyond message field.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with opening line, parameter list, and return format. Slightly verbose in return dict description, but overall efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers all needed aspects: purpose, parameters (with defaults and filters), return format. Output schema exists but description already details returns. No gaps for a list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but description adds meaning: explains adom default, partial match for name_filter, and enumerated protocol options for protocol_filter. Adds value beyond schema titles.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'List custom service objects in an ADOM' and defines the resource. Distinguishes from siblings like list_service_groups by focusing on service objects.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this vs alternatives like list_service_groups or list_addresses. Usage is implied for listing services, but not contextualized.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It explains that only specified fields are updated and lists return values. However, it does not disclose side effects, idempotency, or error conditions (e.g., address not found).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is brief with a clear summary, then structured Args and Returns sections. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 6 parameters and no annotations, the description covers the core behavior and return structure. It lacks error handling details but is sufficient for typical use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description compensates with an Args section detailing each parameter's role (e.g., 'name: Current address name'). This adds meaning beyond the schema's types, though format details for subnet or fqdn could improve clarity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update an existing address object.' and specifies that only provided fields will be modified. It distinguishes from siblings like create, delete, get, list, and update_address_group.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for modifying an existing address but does not explicitly state when to use it versus alternatives, nor does it mention prerequisites like the address must exist. It lacks when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It describes the tool's purpose (adding a model device) but does not disclose behavioral traits like permissions needed, side effects, or that it does not add an actual device. The description is adequate but lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: introductory sentence, conceptual explanation, then clear Args, Returns, and Example sections. Every part is useful and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (mentioned in context), the description still includes a helpful Returns section. It covers purpose, parameters, returns, and an example. Could mention prerequisites like ADOM existence, but otherwise complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage, but the description provides clear explanations for all 6 parameters in the Args section, including default values and an example serial number. This adds significant meaning beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Add a model device for offline provisioning' and explains that model devices allow pre-configuring policies and settings before the actual device connects. This distinguishes it from sibling tools like add_device (adds actual device) and add_devices_bulk.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions usefulness for zero-touch provisioning workflows but does not explicitly say when not to use this tool or compare it directly with alternatives like add_device. Usage context is implied but not fully explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It describes creation, required parameters, and return format, but lacks details on idempotency, duplicate handling, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with Args, Returns, and Example sections. It is concise and front-loaded with the main purpose, earning its place without waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (5 parameters, output schema), the description covers inputs and outputs well. It lacks prerequisites (e.g., ADOM existence) but is largely complete for a creation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description provides clear explanations for all parameters (adom, name, start_ip, end_ip, comment) and includes an example, adding significant meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create an IP range address object.' and distinguishes from siblings like create_address_fqdn, create_address_host, etc., which are listed among sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains what the tool does but does not provide explicit guidance on when to use it vs. alternatives (e.g., create_address_host). Usage context is implied but not stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description provides basic behavioral info (creates, returns dict) but lacks details on side effects, prerequisites, or failure modes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear one-line summary, Args, Returns, and an Example, with no unnecessary information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description explains parameters and return value well, but lacks mention of prerequisites (e.g., ADOM must exist) or edge cases like duplicate names.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Since schema description coverage is 0%, the description fully compensates by listing each parameter with type, default, and meaning in the Args section.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a new policy package' and explains what policy packages contain, distinguishing it from clone_package or delete_package.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies it is for creating new packages but does not explicitly state when to use it vs. alternatives like clone_package or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It correctly implies a read-only operation ('Get details') without side effects, but it does not disclose authorization requirements or any potential behavioral nuances. The description is minimal but accurate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise and front-loaded. It states the purpose in one line, then lists two parameters with brief explanations. Every sentence is essential, and there is no redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple 'get' operation, the description is complete. It identifies the required parameters and indicates a return value. Since an output schema exists, the description need not detail return fields. The tool's complexity is low, and the description covers all necessary information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema coverage is 0%, meaning the description must compensate. It adds semantic meaning by labeling the parameters as 'ADOM name' and 'CLI template group name', which clarifies their purpose beyond the schema's bare type definitions. However, it does not elaborate on expected formats or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Get details of a CLI template group.' It uses a specific verb 'get' and resource 'CLI template group', and it distinguishes from sibling tools like create_cli_template_group, delete_cli_template_group, and list_cli_template_groups by its focus on fetching details of a single group.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide explicit when-to-use or when-not-to-use guidance, nor does it mention alternatives. While the purpose is clear, there is no context about when to prefer this over list_cli_template_groups or other tools. The usage is implied but not explicitly guided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It clearly states the tool is a read operation ('Get detailed information') and outlines the return structure including status and error handling. It does not contradict any annotations, and the behavior is transparent for a get tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: a one-line purpose, followed by clearly labeled Args and Returns sections. Every sentence adds value, and there is no unnecessary text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 required params, no enums, and an output schema), the description is complete. It explains the return format in detail, specifying keys like status, group, and message. No critical information is missing for an agent to use this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, but the description's Args section adds meaningful context: 'adom: ADOM name' and 'name: Service group name'. This goes beyond the bare property names and clarifies the expected values, providing high value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Get detailed information about a service group.' It uses a specific verb ('Get') and a well-defined resource ('service group'). Among sibling tools, there are other get_* tools and list_service_groups, but this one is distinct in that it retrieves details for a single group by name.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like list_service_groups or get_service. It lacks explicit context for usage, such as prerequisites or mention of alternative tools for listing or retrieving other objects.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description discloses the tool's read-only nature and return format including status, count, devices, and message. It provides an example but does not mention error conditions, permissions, or pagination. Overall, it effectively communicates behavior beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with separate Args and Returns sections and an example. It is slightly verbose with redundant context about FortiManager, but the core purpose is front-loaded. Efficient for a two-parameter tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (two optional params, no output schema provided), the description covers purpose, parameters, return format, and an example. It is complete enough for agent invocation but lacks error handling details and prerequisites.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 0%, so the description compensates by explaining the 'adom' default behavior and 'fields' purpose. However, it does not list available field names or validate input types beyond the schema. Adds meaningful context but could be more specific.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists all managed devices in an ADOM, with a specific verb and resource. It distinguishes from sibling tools like list_adoms or list_device_groups by specifying 'devices' and 'managed devices' context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for listing devices but does not explicitly state when to use this tool over alternatives like search_devices or get_device. It lacks when-not-to-use guidance or comparisons with similar list tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It describes the return format and gives an example, but does not explicitly state that the operation is read-only or discuss any side effects. Given the lack of annotations, a score of 3 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with 'Args' and 'Returns' sections and an example. It is slightly verbose but each part adds value. Could be trimmed slightly but still efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the return format with status, count, vdoms, and message, and gives an example. Given the presence of an output schema (implied by the returns documentation), the description is fairly complete for this tool's purpose. Could mention possible error scenarios or rate limits, but not essential here.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must add meaning. It explains 'device' as 'Device name' and 'adom' as 'ADOM name (default: from DEFAULT_ADOM env var, or "root")', which goes beyond the schema's type-only information. This adds useful default and source context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List VDOMs') and the target resource ('for a specific device'). It defines VDOMs and distinguishes itself from sibling list_* tools by focusing on VDOMs. The example reinforces the purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool (to list VDOMs for a device) and provides default behavior for the adom parameter. However, it doesn't explicitly state when not to use it or mention alternatives, though the context among sibling tools makes it clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It describes the return format and mentions status, error messages, and the default ADOM, but does not disclose side effects, authentication needs, or rate limits. The behavior is adequately described for a list operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a summary, context, Args, Returns, and an Example. The explanation of policy packages is slightly verbose but adds useful context. Overall, it is concise and well-organized.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (list with one optional parameter) and the presence of an output schema (implied), the description covers all necessary aspects: what it does, what parameters to use, and what is returned, including error handling and an example.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, but the description adds full semantic meaning for the 'adom' parameter, including default behavior (from env var or 'root'). This compensates completely for the missing schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List all policy packages in an ADOM' with a specific verb and resource, and explains what policy packages are. This distinguishes it from sibling tools like create_package or get_package.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by stating it lists all packages in an ADOM, but does not explicitly provide guidance on when to use it versus alternatives (e.g., get_package for a single package) or mention any exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description states the tool releases a lock, which implies mutation. With no annotations, it carries the burden of transparency. It does not mention authentication requirements, rate limits, or what happens if the ADOM is already unlocked, leaving some behavioral details unclear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two sentences plus structured Args and Returns sections. It is front-loaded with the main purpose. Could be slightly more compact, but overall efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter, the description covers the action, a prerequisite (commit), and the return format. It is reasonably complete given the tool's simplicity and the presence of an output schema description. Minor omissions like checking lock status don't severely impact completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter 'adom' has no schema description (0% coverage). The description adds meaning: 'ADOM name to unlock', which clarifies what the parameter refers to beyond just the type and title. This compensates for the schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'unlock' and the resource 'ADOM (workspace mode)', making the tool's purpose immediately clear. It is distinct from siblings like 'lock_adom' and 'commit_adom'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear usage instruction: 'Changes should be committed before unlocking to persist them.' This tells the agent when to use the tool (after committing). However, it does not explicitly list alternatives or when not to use it, missing some guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses that FQDN addresses are resolved dynamically, which is a key behavioral trait. However, it omits details on permissions, side effects, or return value behavior beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized with a docstring structure: summary, behavioral note, Args, Returns, and Example. Every sentence adds value, and it is appropriately sized with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has an output schema, and the description includes the return structure. It also mentions dynamic resolution behavior. For a create function, this covers essential context, though error handling and prerequisites are not addressed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It lists all parameters with brief explanations, e.g., fqdn: 'Domain name (e.g., "www.example.com")'. This adds meaningful context beyond the raw schema property types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create an FQDN ... address object') and the resource (FQDN address object). Among siblings like create_address_host and create_address_subnet, it uniquely identifies its focus on fully qualified domain names.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description notes that FQDN addresses are resolved dynamically, which guides when to use this tool. It provides a concrete example but does not explicitly contrast with alternatives or list when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It describes the return format (dict with status, package, message) and implies a read-only operation ('Get'), but does not explicitly state safety traits, auth needs, or potential side effects. The description adds some context but lacks thorough behavioral disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured with a purpose line, Args section, and Returns section. Every sentence adds value, with no redundancy. It is front-loaded with the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (3 parameters, 1 required, no enums, no nested objects) and the presence of a described output schema, the description covers all necessary aspects: parameters, defaults, return structure. It is complete for a retrieval tool and matches the context signals.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With schema description coverage at 0%, the description fully compensates by explaining each parameter: name (package name), adom (ADOM name with default from env var or 'root'), and include_details (include policies and settings, default False). This adds significant meaning beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Get detailed information about a policy package.' It uses a specific verb ('Get') and resource ('policy package'), distinguishing it from sibling tools like list_packages (which lists all packages) and delete_package/update_package (which modify).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving details of a single package, but does not explicitly compare with alternatives like list_packages or mention when not to use it. No exclusions or prerequisites are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description must disclose behavior. It mentions listing all script executions and optional device filtering, but does not elaborate on scope (e.g., time range, pagination, permissions), leaving some ambiguity about 'all'.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short, front-loaded with the main purpose, and structured with Args and Returns sections. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and the presence of an output schema (not shown), the description covers the essential return fields. However, it lacks details on pagination or error handling, which could improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description compensates by naming the parameters 'adom' and 'device' and briefly explaining them. The explanation is sufficient for usage but minimal.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'script execution log summary'. It further explains it lists all script executions with log IDs for detailed retrieval, distinguishing it from similar tools like get_script_log_latest or get_script_log_output.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides context that the log IDs are for detailed retrieval, implying a workflow with another tool (get_script_log_output), but it does not explicitly name alternatives or state when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses the return format (dict with status, count, devices, message) and parameter effects. However, it does not mention pagination, rate limits, or whether the search is case-sensitive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is structured with Args and Returns sections and includes examples. It is slightly verbose but appropriate for the complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters and no annotations, the description covers all parameters and return values. It lacks prerequisites or constraints (e.g., ADOM access needed).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description compensates fully. Each parameter is explained with examples, default behavior, and valid values (e.g., connection_status: 'up' or 'down').
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Search for devices with filters.' The name and description indicate a filtered search, distinguishing it from a simple list (list_devices).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Examples are provided but no explicit guidance on when to use this versus alternatives like list_devices or get_device. Usage is implied through examples.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description carries full burden. It states a read-only check ('validates', 'checks') and mentions async behavior (returns Task ID). Does not explicitly confirm non-destructive, but enough context is given.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is very concise: a one-sentence summary followed by bulleted Args and a Returns note. No fluff, information is front-loaded and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Describes core functionality and return type (Task ID). Does not mention prerequisites like device or template group existence, but given 4 parameters and an output schema, the description is reasonably complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must compensate. It lists parameters with generic qualifiers ('ADOM name', 'Template group name') but adds no extra constraints or format details. Barely above bare schema naming.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Validate' and identifies the resource 'template group for a device'. It explains the check (metadata variables resolved), distinguishing it from siblings like 'assign_template_group' or 'get_template_group'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use (pre-deployment validation) but does not explicitly state when not to use or provide alternatives. The context is clear but lacks exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It clearly describes the creation behavior, input parameters, and return format. It does not explicitly state idempotency or whether duplicates are allowed, but the examples and return schema provide sufficient context for a standard create operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is thorough but fairly long due to extensive examples. While well-structured with a clear flow (Args, Returns, Example), some redundancy exists in examples. It is acceptable but could be more concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema is provided, the description covers all necessary aspects: parameter details, return format, and examples. It does not mention creation constraints like unique names or ADOM limitations, but the examples and standard service object concept imply these.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, meaning the description must fully compensate. It does so by explaining each parameter (tcp_portrange, udp_portrange, etc.) with format examples like '80', '8080-8090', '80 443 8080'. This adds significant meaning beyond the schema's type definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a TCP/UDP/SCTP/UDP-Lite service object.' This is a specific verb+resource combination that distinguishes it from sibling tools like create_service_icmp or create_service_group.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes multiple examples showing when to use TCP, UDP, SCTP, and UDP-Lite parameters. It implicitly guides the agent by demonstrating different use cases. However, it does not explicitly state when not to use this tool (e.g., for ICMP services), but the sibling context and name provide differentiation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It warns that the operation cannot be undone and provides a detailed return structure, disclosing behavioral traits beyond the input schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured with a main statement, warning, Args, Returns, and an example. Every section adds value without unnecessary text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers parameters and returns with an example, but lacks mention of prerequisites like ADOM locking or permissions. Given the output schema exists, it is fairly complete but misses minor context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, but the description fully compensates by explaining each parameter (adom, package, policyids) with clear semantics, adding meaning beyond types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool deletes multiple firewall policies at once, with a specific verb and resource, and distinguishes from sibling tools like delete_firewall_policy (single) and delete_devices_bulk (different resource).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The warning about irreversibility provides usage context, but the description does not explicitly state when to use this tool versus alternatives (e.g., delete_firewall_policy for single deletions), nor when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavior. It mentions querying the device through proxy and describes the return format, but does not explicitly state it is read-only or discuss latency or permissions, which would be helpful.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with sections (summary, Args, Returns, Example) and front-loaded purpose. It is slightly lengthy but every part contributes value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter tool, the description covers purpose, parameters, return format, error handling, and an example. With an output schema present, the return description is sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description provides clear descriptions for both parameters ('adom: ADOM name', 'device: Device name') and includes an example usage, adding significant meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Get interface information from a managed device.' It specifies the verb and resource, and distinguishes itself from sibling tools by explaining it queries the actual device through FortiManager proxy for current configuration and status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides context on when to use the tool (to get live interface data) and includes an example. While it does not explicitly exclude alternatives, the focus on real-time query differentiates it from other get/list tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full disclosure burden. It reveals that the tool queries the actual device through FortiManager proxy, returns success/error status, and includes error messages. However, it does not discuss potential slowness, permissions, or rate limits, which would elevate it to a 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear opening, separate sections for args, returns, and an example. Every sentence is necessary, and the front-loaded purpose sentence immediately conveys the tool's function. No superfluous content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 parameters, no nested objects) and the presence of an output schema, the description sufficiently covers behavior, return format, and error handling. The example further clarifies usage, making it complete for an agent to invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no descriptions (0% coverage), but the description's 'Args' section provides brief yet meaningful definitions: 'adom: ADOM name', 'device: Device name'. This adds value beyond the schema's bare types, though more detail (e.g., valid patterns) would be beneficial.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Get real-time status from a managed device' via FortiManager proxy, specifying real-time data like CPU, memory, uptime. This distinguishes it from sibling tools like get_device_status or get_system_status by emphasizing the real-time, proxy-mediated nature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for obtaining current device status but lacks explicit guidance on when to use this tool versus alternative status tools (e.g., get_device_status) or when not to use it. It does not mention prerequisites or limitations, leaving the agent to infer usage from context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations present; description explains the resolve parameter's effect, return format (dict keys), and provides example usage. It does not discuss authorization or rate limits, but for a read operation, this level of detail is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with summary, detailed description, Args, Returns, and Example sections. Slightly verbose but each part serves a purpose; could be tightened but overall effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given there is an output schema (implied by context), the description still adds return value details. Covers all aspects: what tool does, how parameters affect behavior, and expected output. No gaps for this tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description carries full burden. It fully explains all four parameters (adom, package, policy_id, resolve) with types, defaults, and behavior, adding significant value beyond the schema's property names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves services configured on a firewall policy, with optional resolution. This distinguishes it from sibling tools like get_firewall_policy, get_service, or list_services.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description mentions policy hardening workflows as a use case but does not explicitly state when not to use it or provide alternatives. Implicit usage is clear but lacks exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the return structure, example usage, and implied read-only nature. However, it does not mention any side effects, authentication needs, or error handling details beyond a simple error message in the dict.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, front-loaded with the main purpose, followed by bullet points listing returned information, then a clear return schema and an example. Every element serves a purpose without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters, output schema provided), the description is fully complete. It explains what is returned, the structure, and includes a usage example, leaving no ambiguity for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the schema coverage is trivially 100%. The description does not need to add parameter meaning, but it provides detailed return value semantics and an example, which adds value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves FortiManager system status and version information, listing specific fields like version, hostname, serial number, etc. It distinguishes itself from siblings like get_ha_status or get_device_status by focusing on the overall system.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains what the tool does but does not explicitly state when to use it versus alternatives or mention prerequisites. It lacks guidance on when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided. The description makes it clear this is a read operation ('Get detailed status'), and details the return structure including success/error states. While not exhaustive, it sufficiently conveys the non-destructive behavior and expected output.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured with clear sections: one-liner, Args, Returns, Example. It is front-loaded with the core purpose and uses minimal, efficient language.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has an output schema, and the description covers input, output (with keys), and an example. For a simple retrieval tool, this is comprehensive. The only gap is explicit usage guidelines, but overall it is complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description's Args section explains both parameters: task_id (as 'Task ID number') and include_details (as 'Include task line details (default: False)'). This adds meaningful context beyond the input schema, which only provides types and defaults.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves detailed status of a specific task. This distinguishes it from sibling tools like list_tasks (which lists all tasks) and wait_for_task (which waits for completion), as it focuses on a single task's details.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implicitly suggests use for individual task status via its name and return structure, but lacks explicit guidance on when to use this versus alternatives like list_tasks or wait_for_task. No when-not-to-use context is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description adequately discloses behavior: it lists all device groups, is read-only, and returns a defined structure. The return format is described in detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured with a brief intro followed by args and returns. No wasted words, but could be slightly more compact.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with one optional parameter and an output schema, the description is complete. It explains inputs, outputs, and provides enough context for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter adom has no schema description, but the description adds crucial context: default behavior (from env var or 'root'). This compensates fully for the 0% coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists all device groups in an ADOM, with a specific verb and resource. It also explains the purpose of device groups, distinguishing it from list_devices and other list tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions the adom parameter but does not provide explicit guidance on when to use this tool versus alternatives. Usage is implied but not explicitly stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It clearly states the tool is a non-destructive preview, shows what changes would be made, and describes the return structure. It does not mention authentication or rate limits, but these are less critical for a preview tool. The async nature is hinted through the example but not explicitly stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a summary, parameter docs, return block, and example. It is not overly verbose; each section adds value. Slight improvement possible by removing the example's explicit wait_for_task import, but overall efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and moderate complexity, the description covers the tool's purpose, parameters, return schema, and a usage example. It connects to sibling tools (install_package, get_preview_result) implicitly through the workflow. It could explicitly mention the async nature and error handling, but the provided information is largely sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds significant meaning beyond the schema: it explains the purpose of each parameter (adom: ADOM name, package: optional, devices: list of dicts with name and vdom). This compensates for the 0% schema coverage. However, there is a minor contradiction: the schema marks package as required, but the description says 'optional, preview device settings if None', which could confuse agents.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: to preview installation changes before applying, without actually installing. It uses specific verbs and resources (preview, installation changes, devices) and distinguishes itself from sibling tools like install_package and get_preview_result through the description and example workflow.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Use this to verify changes before deployment', indicating when to use. The example workflow further clarifies the sequence (preview, wait for task, get results). However, it does not explicitly exclude alternatives or state when not to use, which would improve guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It describes the return dictionary (status, device, task_id, message) and provides examples. It could disclose more about error handling or what happens if a device already exists, but overall it is transparent about core behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with Args, Returns, and Examples sections. It is informative without being overly verbose, though it could be slightly more concise by merging some explanatory lines.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (10 parameters, no annotations) and the presence of an output schema, the description is comprehensive. It covers device types, required vs optional parameters, return format, and examples. Minor omission: it does not explain possible values for mgmt_mode beyond the default.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description must add meaning for all parameters. It thoroughly explains each parameter (adom, name, ip, serial_number, admin_user, admin_pass, description, platform, mgmt_mode, flags) with defaults and examples, far exceeding basic schema information.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Add a new device to FortiManager' and distinguishes between adding a real device (with IP) and a model device (with serial number). This effectively differentiates it from sibling tools like add_model_device and add_devices_bulk.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on when to use the tool by detailing two use cases (real device with IP, model device with serial number) with examples. However, it does not explicitly state when not to use it or mention alternatives like add_model_device, which could lead to slight ambiguity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description fully explains what the tool does: creates an address group, lists parameters, and specifies the return format. It could mention idempotency or error cases, but overall behavior is clear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with clear sections for purpose, args, returns, and an example. Every sentence adds value, and it is concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity and the presence of siblings, the description fully covers what an agent needs to know: input requirements, output format, and a usage example. It is complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description compensates by explaining each parameter (adom, name, members, comment) and provides a concrete example. This adds significant meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create an address group' and distinguishes it from siblings like create_address_fqdn or create_address_subnet by specifying it groups address objects. The example reinforces the purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for creating address groups but does not explicitly state when to use this tool versus other address creation tools. No alternatives or exclusions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the full burden. It clearly explains that this tool creates an ICMP service, defines all parameters with ranges, and specifies the return format (dict with status, name, message). It does not disclose potential side effects or permissions, but for a simple create operation, this is adequate. No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is concise and well-structured with clear sections (Args, Returns, Example). The first sentence immediately states the purpose. No unnecessary words. Suitable length for the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's parameter count (5) and lack of annotations, the description covers inputs, outputs, and provides an example. The output schema exists but description still explains return fields. This is complete for an AI agent to understand and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. Each parameter is described: adom as 'ADOM name', name as 'Service object name', icmp_type with range '0-255, optional for all types', icmp_code with range '0-255, optional', and comment as 'Optional comment'. The example adds practical context (icmp_type=8 for ping). This fully explains parameter semantics beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Create an ICMP service object' with a specific verb and resource. The example (ping with ICMP type 8) further clarifies. This distinguishes from siblings like create_service_tcp_udp and create_service_group, which are for other service types.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for ICMP service objects but does not explicitly state when to use this tool vs alternatives like create_service_tcp_udp. No 'when to use' or 'when not to use' guidance is provided. Sibling tools exist for TCP/UDP and service groups, but no comparison is made.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses the destructive nature ('cannot be undone'), that it does not affect the actual device, and describes the return structure. It lacks details on auth needs or rate limits, but provides sufficient context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: brief one-liner, paragraph, Args/Returns sections, and an example. Every sentence adds value with no fluff, making it efficient and easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and sibling tools, the description covers purpose, effect, parameters, return values, and provides an example. It is fairly complete, though it could mention what happens if the device does not exist.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description adds meaning by explaining each parameter: adom, device, and flags (with example). This compensates for the schema's lack of descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete a device from FortiManager' and explains the scope: 'Removes a device registration. Does not affect the actual device or its configuration.' This distinguishes it from sibling tools like add_device or get_device.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool (to remove a device from FortiManager management) and warns that it cannot be undone. It does not explicitly mention alternatives or prerequisites, but given the sibling tools, no other deletion tool exists for devices.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description bears full burden. It describes the return structure (dict with status, count, devices, message) and gives example usage. It implies a read-only operation but doesn't explicitly state side effects or auth needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is well-structured with bullet points and docstring-style details. It is slightly lengthy but front-loaded with purpose. Could be trimmed, but overall efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given output schema exists, description explains return values effectively. For a tool with two optional parameters, it covers usage, defaults, and return format comprehensively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description must compensate. It explains both parameters: adom (default from env var or 'root') and device (optional, returns all if not given). This adds meaning beyond the schema's bare structure.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves device status including connection and config sync status. It lists specific status fields and distinguishes from sibling tools like get_device_realtime_status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains how to get status for all devices or a specific device via the 'device' parameter, and mentions the 'adom' default. However, it lacks explicit guidance on when to use this tool versus alternatives like get_device or get_device_realtime_status.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses that the tool returns detailed configuration changes with status, preview data, and error message, implying a read-only operation. No side effects are mentioned but appropriate for a retrieval tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is structured with an intro, Args, and Returns sections. It is concise but includes a detailed Returns description. Could be slightly more front-loaded but overall efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given an output schema is provided (Returns section), the description fully explains the return structure and the tool's role as a follow-up to preview_install. It is complete for the task.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must compensate. It explains adom as 'ADOM name' and devices with an example format, adding meaning beyond the schema's minimal type definitions. However, it does not fully describe the expected keys in the device objects.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Get the result of an installation preview' and specifies it is to be called after `preview_install()`, distinguishing it from sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly instructs to call after preview_install() task completes and explains the purpose, but does not mention when not to use or provide alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the behavioral burden. It specifies that the tool returns a dict with status, count, addresses, and message, but does not explicitly state it is a read-only operation or disclose any side effects, rate limits, or authentication requirements. The term 'list' implies safety, but explicit disclosure is missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with Args and Returns sections, concise yet comprehensive. Every sentence adds value, and it includes practical examples. It is appropriately front-loaded with the main purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (3 parameters, output schema exists), the description is complete. It explains the return dictionary structure, parameter semantics, and provides usage examples. No obvious gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description fully compensates by explaining each parameter: adom (ADOM name with default from env var), name_filter (partial match filtering), and type_filter (allowed values: ipmask, fqdn, iprange, wildcard). It also includes default behavior and usage examples.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List firewall address objects in an ADOM' and defines address objects as network entities (hosts, subnets, FQDNs). This distinguishes it from sibling tools like get_address (singular), create_address_*, delete_address, and list_address_groups.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context with parameter explanations and examples (e.g., using type_filter='fqdn' to find FQDN addresses). However, it does not explicitly state when not to use this tool or suggest alternatives like search_objects or get_address.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It details the return dict structure (status, count, total, policies, message) and pagination parameters (limit, offset) with examples. However, it does not explicitly state that the operation is read-only or safe, nor does it mention authorization requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized with Args, Returns, and Example sections. Every sentence is informative, no redundancy, and it is appropriately sized for the complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 5 parameters, no annotations, and an output schema (though not detailed), the description covers input semantics, output structure, and provides examples. It fully addresses the agent's needs for a list operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description fully explains each parameter (adom, package, fields, limit, offset) with context and usage examples. It adds significant value beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List firewall policies in a policy package,' specifying the verb (list), resource (firewall policies), and scope (policy package). This distinguishes it from siblings like get_firewall_policy (single), search_firewall_policies (filtered), and mutation tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives like search_firewall_policies or get_firewall_policy. The description implies it lists all policies in a package, but does not clarify use cases or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes return format and filter behavior, but does not mention potential limits (e.g., pagination) or side effects. Without annotations, the description carries the burden; it's adequate but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized with Args, Returns, and Example sections. It is concise with no redundant information, fitting the tool's simplicity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the low complexity and presence of an output schema, the description covers all needed aspects: purpose, parameter, return format, and an example. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema coverage, the description fully documents the sole parameter filter_state, listing all valid enum values with clear descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List all tasks in FortiManager' and explains tasks as background operations, distinguishing it from siblings like get_task (single task) and wait_for_task (waiting).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides usage context with optional filter_state parameter and an example. Lacks explicit when-not-to-use or alternatives, but the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries burden. It discloses the adom default behavior and limit parameter, and the operation is inherently read-only. However, it does not explicitly state safety or side effects, but the simplicity justifies a high score.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured with Args and Returns sections. Every sentence earns its place without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the description adequately covers purpose, parameters, and return. No additional context needed for a list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description adds significant meaning: it explains adom default from env var and limit as maximum number to return, far exceeding the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List template groups in an ADOM' and explains what template groups are, distinguishing them from individual templates. Sibling tools like list_templates and list_cli_template_groups confirm differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides basic usage context but does not explicitly compare to alternatives or state when not to use this tool. It implies usage for listing but lacks exclusion guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description is transparent about being a read-only listing operation. It does not contradict any annotations (none provided) and discloses the return format. For a simple list tool, this is adequate behavioral disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: a one-line summary, followed by details in an Args/Returns format. Every sentence adds value, and there is no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the tool's purpose, parameters with defaults, and return format. Given it's a list tool with an output schema, it provides sufficient context for correct invocation, though it could mention whether results are paginated or ordered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema coverage, the description adds meaningful context for both parameters: adom (with default behavior) and limit (maximum count). It also explains the return structure. This compensates well for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists all provisioning templates in an ADOM, returning all types. It distinguishes itself from siblings like list_sdwan_templates and list_system_templates by specifying it covers all types, making its scope immediately clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains it returns all template types, implying it's the general list tool. However, it does not explicitly state when to use it over type-specific lists (e.g., if only IPsec templates are needed). Still, the context provides enough guidance for appropriate use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description carries full burden. It discloses that filters perform partial matches, describes the return format (dict with status, count, policies, message), and includes examples. It does not mention side effects (likely none as it's read-only) or rate limits, but is fairly transparent for a search tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a purpose statement, Args list, Returns, and Examples. It is front-loaded and clear, though slightly lengthy. Every sentence adds value, and the structure aids readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (8 parameters, 2 required) and the presence of an output schema (not shown), the description adequately covers parameter semantics and return format. It does not mention error handling beyond a message field or rate limits, but is sufficient for a search tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description must compensate. It lists all 8 parameters with clear explanations (e.g., 'Filter by policy name (partial match)'), adding significant meaning beyond the schema which only provides names and types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Search firewall policies with filters.' It uses a specific verb (search) and resource (firewall policies), and distinguishes from sibling tools like list_firewall_policies (which lists all) and get_firewall_policy (which gets a specific one).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on when to use the tool (when filtering is needed) via examples, but does not explicitly state when not to use it or mention alternatives. The examples are helpful but lack explicit exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It describes the return format in detail (status, lists of matches, total count, message) and notes partial matching. It does not mention side effects or error handling but is otherwise transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with sections (Args, Returns, Example) and front-loads the main purpose. It is slightly verbose due to the detailed Returns dict, but every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (searches multiple object types) and the presence of an output schema, the description fully explains the output format and includes an example. It covers all necessary aspects.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, but the description's Args section adds meaning: 'ADOM name' and 'Search term (partial match)'. This is sufficient, though more detail could be added.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool searches across all firewall objects by name, listing specific object types (addresses, address groups, services, service groups). This distinguishes it from sibling search tools like search_devices and search_firewall_policies.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use this tool (searching firewall objects by name). However, it does not explicitly mention when not to use it or compare it to alternative search tools among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly states that the policy is not active until installed, explains the default values for parameters like action, nat, logtraffic, and indicates that policyid is optional and auto-assigned. It does not discuss permissions or other side effects, but covers the key behavioral trait.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized with separate sections for Args, Returns, and an Example. It is front-loaded with the main purpose. While slightly long, every sentence provides value, and the structure aids readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (15 parameters, 8 required) and the presence of an output schema description in the Returns section, the description is complete. It covers the creation process, defaults, optional parameters, and the response format, leaving no critical gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, so the description completely compensates by providing detailed explanations for all 15 parameters, including examples, defaults, and allowed values. This adds significant meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a new firewall policy, specifies the resource (policy in a package), and differentiates from siblings like update_firewall_policy, delete_firewall_policy, etc. The verb 'create' is specific and the scope is well-defined.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions that the policy won't be active until the package is installed, providing important usage context. However, it does not explicitly state when to use this tool versus alternatives like update_firewall_policy or delete_firewall_policy, though the distinction is implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses destructive nature (cannot be undone), return format, and task mechanism via flags, though lacks permissions or prerequisites.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise yet complete: includes warning, parameter descriptions, return dictionary, and example. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, parameters, returns, and example. Lacks detail on partial failure or concurrency, but sufficient for a delete tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds substantial meaning to all three parameters beyond schema: explains adom, devices list, and flags with example value 'create_task'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it deletes multiple devices in bulk, distinguishing from singular delete_device and bulk add tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context for bulk deletion with a warning about irreversibility, but does not explicitly mention when not to use or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses the return structure (status, adom, message) and includes an example. While it does not explicitly state read-only behavior, the verb 'Get' implies it. Describes error handling via message key.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is well-structured with Args, Returns, and Example sections. Every sentence adds value, and the content is appropriately sized for the tool's simplicity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description provides additional context via an example and error handling details. For a simple 2-parameter tool, the description is complete and leaves no ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It does so effectively: explains 'name' with examples ('root', 'customer-a') and 'include_details' with 'Include sub-objects (default: False)'. This adds significant meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Get detailed information about a specific ADOM' using a specific verb and resource, and the tool is distinct from sibling get_* tools that target different resources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implicitly indicates when to use (to get ADOM details) but does not provide explicit when-not-to-use or alternatives. The context of sibling tools provides natural differentiation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries full responsibility for behavioral disclosure. It clearly states that the operation is asynchronous, requires a preview before actual install, and that preview authorizes a single install only when the package is unchanged. It also explains the preview parameter behavior and what the return value contains (status, task_id, message).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a summary line, a behavioral paragraph, parameter list, return values, and examples. It is slightly verbose but every part adds value. The examples are helpful. Could be slightly more concise, but still effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers all necessary aspects: purpose, usage workflow, parameters, return values, and examples. With an output schema present, the description still provides sufficient context about return values. It is complete for an agent to correctly select and invoke the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description compensates by explaining each parameter: adom, package, devices (with example format), and preview (default False, preview-only behavior). The device parameter format is shown in the example and args section, adding meaning beyond the schema types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the tool's purpose: 'Install a policy package to managed devices.' It specifies the action (install) and the resource (policy package), and distinguishes it from sibling tools like preview_install and assign_* tools by focusing on deployment of firewall policies from a package to devices.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear guidance on when to use the tool, including the prerequisite of a verified preview via preview_install and the behavior of the preview parameter. It also explains the asynchronous nature and need to use wait_for_task. However, it does not explicitly mention alternatives or when not to use it, such as for assigning templates.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description carries full burden. It discloses the basic behavior (locks ADOM, returns status/message) and includes a usage pattern. It does not specify idempotency or error handling for already locked ADOMs, but for a simple lock tool, the description is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficient: a brief one-sentence purpose, then structured Args/Returns/Example. Every part adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and the presence of sibling unlock_adom and commit_adom, the description provides a complete workflow. The return format is defined, and the example illustrates typical usage. No output schema needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage, and the description adds clear meaning: 'ADOM name to lock'. The single parameter is well-defined, and the example shows usage with 'root'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Lock an ADOM for editing' using a specific verb and resource. It distinguishes from siblings like unlock_adom by specifying 'workspace mode' and the prerequisite for changes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use: 'In workspace mode, ADOMs must be locked before making changes' and hints at conflict prevention. It provides an example that includes commit_adom and unlock_adom, implying a workflow, but does not explicitly state when not to use or list alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description provides key behavioral details: it reorders policies and returns a status/error dict. It could mention potential side effects like policy validation, but the explanation is adequate for a reordering tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: a concise summary line, a brief explanation, then organized sections for Args, Returns, and an Example. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple move operation with no output schema, the description covers purpose, parameters, return format, and usage example. It is self-contained and sufficient for an agent to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 0% schema coverage, the description thoroughly documents all five parameters in Args, including default for 'position', and provides a concrete example. This adds significant meaning beyond the basic schema types and titles.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Move a firewall policy to a new position') and the resource ('firewall policy'). It explains that reordering changes evaluation priority, which distinguishes it from other policy tools like create, update, or delete.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use this tool (to reorder a policy relative to another) and how position affects priority. While it doesn't explicitly state when not to use it or list alternatives, the context is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that it forces a cache refresh, implying a write operation with potential side effects. It mentions return status and message, but does not detail permissions or impacts. Given no annotations, this is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear title, usage note, and separate sections for args and returns. Every sentence adds value; no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers purpose, usage context, parameter semantics, and return structure. It is fully self-contained and complete for a simple tool with one optional parameter.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds significant value beyond the schema by explaining the adom parameter's default (from DEFAULT_ADOM env var or 'root'). With 0% schema coverage, this is essential and well-provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it reloads the device list from FortiManager database, forcing a cache refresh. It distinguishes from siblings like list_devices and add_device by focusing on refreshing existing data rather than listing or adding.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly says 'Useful after direct database changes or if device list appears stale', providing clear when-to-use guidance. However, it does not mention when not to use or explicitly compare with alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It explains polling until completion or timeout, includes timeout cap at 3600s, and describes the return dict. It does not detail error behavior or side effects, but the output schema covers return states. Transparency is high.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with Args, Returns, and Example sections. It is concise, front-loaded with purpose, and every sentence adds value. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema explains return values, the description covers all needed aspects: parameters, behavior, and usage example. It is complete for a polling utility tool with clear constraints.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains task_id, timeout (with default 300 and cap 3600), and poll_interval (default 5) clearly. This adds significant meaning beyond the schema, which lacks descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Wait for a task to complete' with a polling mechanism. It distinguishes from sibling tools that perform other operations like install, execute, or get. The specific verb 'wait' and resource 'task' make purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description says 'Useful for waiting on installation or provisioning operations,' providing clear context. It suggests when to use this tool, though it could explicitly mention that it should be used after starting an async operation. Still, it gives good guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![fortimanager-mcp MCP server](https://glama.ai/mcp/servers/rstierli/fortimanager-mcp/badges/card.svg)](https://glama.ai/mcp/servers/rstierli/fortimanager-mcp)

Score Badge

Copy to your README.md:

[![fortimanager-mcp MCP server](https://glama.ai/mcp/servers/rstierli/fortimanager-mcp/badges/score.svg)](https://glama.ai/mcp/servers/rstierli/fortimanager-mcp)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/rstierli/fortimanager-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server