OPNSense MCP Server
The OPNSense MCP Server enables Infrastructure as Code (IaC) management of OPNsense firewalls through comprehensive API integration. With this server, you can:
Configure OPNsense Connection: Set up host, API key, and secret for firewall communication
Test Connectivity: Verify API connection and authentication
Manage VLANs: List, create, update, delete, and retrieve VLAN details
Control Firewall Rules: List, create, update, delete, enable/disable, and search rules; supports predefined rule presets
Handle Backups: Create, list, and restore configuration backups
Network Operations: Retrieve available network interfaces and configure isolated networks
DNS Management: Manage DNS blocklists
IaC Integration: Declaratively manage OPNsense infrastructure using JSON or JavaScript
Provides tools for managing OPNSense firewalls, including VLAN creation and management, firewall rule configuration, network interface queries, and DHCP lease management
Implements an audit database for tracking changes made through the OPNSense MCP server
Used as an optional cache layer for improved performance in Phase 3 of the OPNSense MCP server
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@OPNSense MCP Serverlist all firewall rules and show me any that allow traffic from the DMZ"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
OPNsense MCP Server
A Model Context Protocol (MCP) server for comprehensive OPNsense firewall management. This server enables AI assistants like Claude to directly manage firewall configurations, diagnose network issues, and automate complex networking tasks.
Features
š„ Firewall Management
Complete CRUD operations for firewall rules
Proper handling of API-created "automation rules"
Inter-VLAN routing configuration
Batch rule creation and management
Enhanced persistence with multiple fallback methods
š NAT Configuration (SSH-based)
Outbound NAT rule management
NAT mode control (automatic/hybrid/manual/disabled)
No-NAT exception rules for inter-VLAN traffic
Automated DMZ NAT issue resolution
Direct XML configuration manipulation
š Network Diagnostics
Comprehensive routing analysis
ARP table inspection with vendor identification
Interface configuration management
Network connectivity troubleshooting
Auto-fix capabilities for common issues
š„ļø SSH/CLI Execution
Direct command execution on OPNsense
Configuration file manipulation
System-level operations not available via API
Service management and restarts
š Additional Capabilities
VLAN management
DHCP lease viewing and management
DNS blocklist configuration
HAProxy load balancer support
Configuration backup and restore
Infrastructure as Code support
Related MCP server: pfSense MCP Server
Installation
Prerequisites
Node.js 18+ or Bun 1.0+
OPNsense firewall (v24.7+ recommended)
API credentials for OPNsense
SSH access (optional, for advanced features)
Quick Start with npm
Install the package:
npm install -g opnsense-mcp-serverCreate a
.envfile with your credentials:
# Required
OPNSENSE_HOST=https://your-opnsense-host:port
OPNSENSE_API_KEY=your-api-key
OPNSENSE_API_SECRET=your-api-secret
OPNSENSE_VERIFY_SSL=false
# Optional - for SSH features
OPNSENSE_SSH_HOST=your-opnsense-host
OPNSENSE_SSH_USERNAME=root
OPNSENSE_SSH_PASSWORD=your-password
# Or use SSH key
# OPNSENSE_SSH_KEY_PATH=~/.ssh/id_rsaStart the MCP server:
opnsense-mcp-serverQuick Start with Bun (Faster)
Bun provides significantly faster startup times and better performance.
Install Bun (if not already installed):
curl -fsSL https://bun.sh/install | bashClone and install:
git clone https://github.com/vespo92/OPNSenseMCP.git
cd OPNSenseMCP
bun installCreate your
.envfile (same as npm version above)Run with Bun:
# Development with hot reload
bun run dev:bun
# Production
bun run start:bunUsing Bun with Claude Desktop
{
"mcpServers": {
"opnsense": {
"command": "bun",
"args": ["run", "/path/to/OPNSenseMCP/src/index.ts"],
"env": {
"OPNSENSE_HOST": "https://your-opnsense:port",
"OPNSENSE_API_KEY": "your-key",
"OPNSENSE_API_SECRET": "your-secret",
"OPNSENSE_VERIFY_SSL": "false"
}
}
}
}Usage with Claude Desktop (npm)
Add to your Claude Desktop configuration (claude_desktop_config.json):
{
"mcpServers": {
"opnsense": {
"command": "npx",
"args": ["opnsense-mcp-server"],
"env": {
"OPNSENSE_HOST": "https://your-opnsense:port",
"OPNSENSE_API_KEY": "your-key",
"OPNSENSE_API_SECRET": "your-secret",
"OPNSENSE_VERIFY_SSL": "false"
}
}
}
}Common Use Cases
Fix DMZ NAT Issues
// Automatically fix DMZ to LAN routing
await mcp.call('nat_fix_dmz', {
dmzNetwork: '10.0.6.0/24',
lanNetwork: '10.0.0.0/24'
});Create Firewall Rules
// Allow NFS from DMZ to NAS
await mcp.call('firewall_create_rule', {
action: 'pass',
interface: 'opt8',
source: '10.0.6.0/24',
destination: '10.0.0.14/32',
protocol: 'tcp',
destination_port: '2049',
description: 'Allow NFS from DMZ'
});Diagnose Routing Issues
// Run comprehensive routing diagnostics
await mcp.call('routing_diagnostics', {
sourceNetwork: '10.0.6.0/24',
destNetwork: '10.0.0.0/24'
});Execute CLI Commands
// Run any OPNsense CLI command
await mcp.call('system_execute_command', {
command: 'pfctl -s state | grep 10.0.6'
});MCP Tools Reference
The server provides 50+ MCP tools organized by category:
Firewall Tools
firewall_list_rules- List all firewall rulesfirewall_create_rule- Create a new rulefirewall_update_rule- Update existing rulefirewall_delete_rule- Delete a rulefirewall_apply_changes- Apply pending changes
NAT Tools
nat_list_outbound- List outbound NAT rulesnat_set_mode- Set NAT modenat_create_outbound_rule- Create NAT rulenat_fix_dmz- Fix DMZ NAT issuesnat_analyze_config- Analyze NAT configuration
Network Tools
arp_list- List ARP table entriesrouting_diagnostics- Diagnose routing issuesrouting_fix_all- Auto-fix routing problemsinterface_list- List network interfacesvlan_create- Create VLAN
System Tools
system_execute_command- Execute CLI commandbackup_create- Create configuration backupservice_restart- Restart a service
For a complete list, see docs/api/mcp-tools.md.
Documentation
Testing
The repository includes comprehensive testing utilities:
# Test NAT functionality
npx tsx scripts/test/test-nat-ssh.ts
# Test firewall rules
npx tsx scripts/test/test-rules.ts
# Test routing diagnostics
npx tsx scripts/test/test-routing.ts
# Run all tests
npm testDevelopment
Building from Source
git clone https://github.com/vespo92/OPNSenseMCP.git
cd OPNSenseMCP
npm install
npm run buildProject Structure
OPNSenseMCP/
āāā src/ # Source code
ā āāā api/ # API client
ā āāā resources/ # Resource implementations
ā āāā index.ts # MCP server entry
āāā docs/ # Documentation
āāā scripts/ # Utility scripts
ā āāā test/ # Test scripts
ā āāā debug/ # Debug utilities
ā āāā fixes/ # Fix scripts
āāā dist/ # Build outputTroubleshooting
API Authentication Failed
Verify API key and secret are correct
Ensure API access is enabled in OPNsense
Check firewall rules allow API access
SSH Connection Failed
Verify SSH credentials in
.envEnsure SSH is enabled on OPNsense
Check user has appropriate privileges
NAT Features Not Working
NAT management requires SSH access
Add SSH credentials to environment variables
Test with:
npx tsx scripts/test/test-nat-ssh.ts
Contributing
Contributions are welcome! Please see CONTRIBUTING.md for guidelines.
License
This project is licensed under the MIT License - see the LICENSE file for details.
Support
Issues: GitHub Issues
Discussions: GitHub Discussions
Documentation: Full Documentation
Acknowledgments
Built for use with Anthropic's Claude
Implements the Model Context Protocol
Designed for OPNsense firewall
Version: 0.8.2 | Status: Production Ready | Last Updated: August 2025
Available Tools
64 toolsapply_blocklist_categoryC
Apply a predefined category of domain blocks
| Name | Required | Description | Default |
|---|---|---|---|
| category | Yes | Category of domains to block |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. 'Apply' suggests a write/mutation operation, but it doesn't disclose critical behavioral traits: whether this requires admin permissions, if it's reversible (e.g., via 'unblock_domain'), what happens to existing blocks in the category, or if it triggers system restarts. For a mutation tool with zero annotation coverage, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and resource, making it easy to parse. Every word earns its place without redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no annotations and no output schema, the description is incomplete. It lacks information on permissions, side effects, reversibility, and what 'apply' entails operationally (e.g., immediate enforcement, logging). Given the complexity of domain blocking and sibling tools, more context is needed for safe and effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with the single parameter 'category' fully documented in the schema (including enum values). The description adds no additional parameter semantics beyond implying it's for 'predefined' categories, which the schema's enum already suggests. Baseline 3 is appropriate when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('apply') and resource ('predefined category of domain blocks'), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'block_domain' or 'toggle_blocklist_entry', which likely handle individual domain blocks rather than categories.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With siblings like 'block_domain', 'toggle_blocklist_entry', and 'list_dns_blocklist', there's no indication whether this tool is for bulk operations, preset configurations, or differs in permanence. The agent must infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
block_domainC
Add a domain to the DNS blocklist
| Name | Required | Description | Default |
|---|---|---|---|
| description | No | Optional description for the block | |
| domain | Yes | Domain to block (e.g., pornhub.com) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only states the action without behavioral context. It doesn't mention permissions required, whether the change is immediate or requires a restart, if it's reversible via 'unblock_domain', rate limits, or what happens on success/failure. For a mutation tool with zero annotation coverage, this is inadequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that gets straight to the point with zero wasted words. It's appropriately sized for this simple operation and front-loads the essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no annotations and no output schema, the description is incomplete. It doesn't explain what happens after adding (e.g., confirmation message, error conditions), how to verify with 'list_dns_blocklist', or relationship to sibling tools. The agent lacks sufficient context for safe, effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are fully documented in the schema. The description adds no additional parameter semantics beyond what's already in the schema (domain format example, description purpose). Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Add') and target resource ('domain to the DNS blocklist'), making the purpose immediately understandable. It doesn't explicitly differentiate from sibling tools like 'block_multiple_domains' or 'unblock_domain', but the single-domain focus is implied.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided about when to use this tool versus alternatives like 'block_multiple_domains' for multiple domains, 'toggle_blocklist_entry' for enabling/disabling, or 'unblock_domain' for removal. The description only states what it does, not when it's appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
block_multiple_domainsC
Block multiple domains at once
| Name | Required | Description | Default |
|---|---|---|---|
| description | No | Optional description for the blocks | |
| domains | Yes | List of domains to block |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('Block') which implies a write/mutation operation, but doesn't clarify permissions needed, whether the blocks are immediate or require confirmation, if there are rate limits, or what happens on failure. This leaves significant gaps for a tool that modifies system state.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that communicates the core functionality without unnecessary words. It's appropriately sized for a straightforward tool and is completely front-loaded with the essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no annotations and no output schema, the description is insufficient. It doesn't explain what 'blocking' entails operationally, what the expected outcome is, error conditions, or how this interacts with other blocking mechanisms in the system. The context signals show this is a state-changing operation that needs more behavioral context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with both parameters ('domains' and 'description') clearly documented in the schema. The description adds no additional parameter information beyond what's in the schema, so it meets the baseline for high schema coverage without compensating value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Block') and resource ('multiple domains'), making the purpose immediately understandable. It distinguishes from the sibling 'block_domain' by specifying 'multiple domains at once', though it doesn't explicitly contrast with other related tools like 'apply_blocklist_category' or 'toggle_blocklist_entry'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'block_domain' (for single domains) or 'apply_blocklist_category' (for category-based blocking). The description implies batch processing but lacks explicit context about prerequisites, timing, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
configureC
Configure OPNsense connection
| Name | Required | Description | Default |
|---|---|---|---|
| apiKey | Yes | API key | |
| apiSecret | Yes | API secret | |
| host | Yes | OPNsense hostname or IP | |
| verifySsl | No | Verify SSL certificate |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden but lacks behavioral details. It doesn't disclose if this is a one-time setup, if it persists credentials, requires authentication, has side effects, or any rate limits. 'Configure' implies a write operation, but specifics are missing, leaving gaps in understanding.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It's front-loaded with the core action ('Configure'), though it could be more specific. Structure is straightforward, earning a high score for brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is incomplete. It doesn't explain what happens after configurationādoes it return a status, store session data, or enable other tools? For a tool with 4 parameters and potential side effects, more context is needed to guide the agent effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are well-documented in the schema. The description adds no extra meaning beyond implying these inputs are for connection setup. Baseline 3 is appropriate as the schema handles parameter semantics adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the tool configures an OPNsense connection, which is a clear purpose. However, it's vague about what 'configure' entailsādoes it set up credentials, test connectivity, or initialize a session? It doesn't distinguish from siblings like 'test_connection', which might overlap in function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., must be called before other tools), exclusions, or how it relates to siblings like 'test_connection'. This leaves the agent guessing about proper context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_backupC
Create a configuration backup
| Name | Required | Description | Default |
|---|---|---|---|
| description | No | Backup description |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states 'Create a configuration backup' which implies a write operation, but does not cover critical aspects such as permissions required, whether it's idempotent, rate limits, what happens if a backup already exists, or the format/scope of the backup. This leaves significant gaps for an agent to understand the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with just three words ('Create a configuration backup'), front-loaded with the core action, and contains no unnecessary information. It efficiently communicates the essence of the tool without waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a backup creation tool (a write operation with potential side effects), no annotations, and no output schema, the description is insufficient. It lacks details on what the backup entails, how to verify success, error handling, or interaction with sibling tools, making it incomplete for safe and effective use by an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for its single parameter ('description'), so the schema already documents it adequately. The description does not add any additional meaning or context about the parameter beyond what the schema provides, such as examples or constraints, but this is acceptable given the high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Create a configuration backup' clearly states the action (create) and resource (configuration backup), making the purpose understandable. However, it does not differentiate from sibling tools like 'list_backups' or 'restore_backup', which would require more specificity about what type of backup or context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'list_backups' or 'restore_backup', nor does it mention prerequisites, dependencies, or typical use cases. It simply states what the tool does without contextual usage information.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_firewall_presetC
Create a firewall rule from a preset
| Name | Required | Description | Default |
|---|---|---|---|
| description | No | Description override (optional) | |
| destination | No | Destination override (optional) | |
| interface | Yes | Interface name | |
| preset | Yes | Preset name | |
| source | No | Source override (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It states 'create' implying a write/mutation operation but doesn't disclose behavioral traits like permissions needed, side effects, whether it's idempotent, or what happens on success/failure. For a tool that creates firewall rules, this is a significant gap in safety and operational context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and resource. Every word earns its place, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is incomplete for a tool that creates firewall rules. It lacks information on behavioral context (e.g., what 'create' entails, error handling), output expectations, and differentiation from siblings. For a mutation tool with security implications, this is inadequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds no additional parameter semantics beyond what's in the schema (e.g., it doesn't explain preset meanings or override behaviors). Baseline 3 is appropriate since the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('create') and resource ('firewall rule from a preset'), making the purpose understandable. It distinguishes from sibling 'create_firewall_rule' by specifying 'from a preset', but doesn't explicitly contrast them. The description avoids tautology by not just repeating the name.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'create_firewall_rule' or other firewall-related tools. The description implies usage with presets but doesn't specify scenarios, prerequisites, or exclusions. This leaves the agent without context for tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_firewall_ruleC
Create a new firewall rule
| Name | Required | Description | Default |
|---|---|---|---|
| action | Yes | Rule action (pass/block/reject) | |
| description | No | Rule description | |
| destination | Yes | Destination address/network or "any" | |
| destinationPort | No | Destination port (for TCP/UDP) | |
| direction | Yes | Traffic direction | |
| enabled | No | Enable rule | |
| interface | Yes | Interface name | |
| protocol | Yes | Protocol (any/tcp/udp/icmp) | |
| source | Yes | Source address/network or "any" | |
| sourcePort | No | Source port (for TCP/UDP) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. 'Create a new firewall rule' implies a write/mutation operation but doesn't specify whether this requires admin permissions, whether rules take effect immediately, if there are rate limits, or what happens on failure. For a security-critical mutation tool, this is insufficient behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is maximally concise at just four words, front-loading the essential information with zero wasted words. Every element earns its place, making it easy to parse while conveying the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a firewall rule creation tool with 10 parameters, no annotations, and no output schema, the description is incomplete. It doesn't address security implications, permission requirements, typical workflows, or what the tool returns. The agent must rely entirely on the input schema without higher-level guidance about this critical infrastructure operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds no parameter information beyond what's already in the schema, which has 100% coverage with detailed descriptions for all 10 parameters. The baseline score of 3 reflects adequate parameter documentation through the schema alone, though the description could have provided higher-level context about parameter relationships or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('create') and resource ('firewall rule'), making the purpose immediately understandable. However, it doesn't distinguish this tool from its sibling 'create_firewall_preset' or explain how it differs from 'update_firewall_rule', which prevents a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'create_firewall_preset', 'update_firewall_rule', or 'delete_firewall_rule'. It also doesn't mention prerequisites, dependencies, or typical use cases for firewall rule creation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_vlanC
Create a new VLAN
| Name | Required | Description | Default |
|---|---|---|---|
| description | No | VLAN description | |
| interface | Yes | Physical interface (e.g., igc3) | |
| pcp | No | Priority Code Point (0-7) | 0 |
| tag | Yes | VLAN tag (1-4094) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. 'Create a new VLAN' implies a mutation operation, but it doesn't disclose behavioral traits such as permissions required, whether it's idempotent, potential side effects (e.g., network disruption), or error handling. This is a significant gap for a mutation tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with a single sentence ('Create a new VLAN'), which is front-loaded and wastes no words. For a tool with a clear name and good schema coverage, this brevity is efficient and appropriate.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a network configuration tool (creating VLANs), lack of annotations, and no output schema, the description is incomplete. It doesn't address key contextual aspects like what happens after creation (e.g., VLAN activation), return values, or error scenarios, leaving gaps for an AI agent to understand the tool fully.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters (description, interface, pcp, tag) with their types and constraints. The description adds no additional meaning beyond what's in the schema, such as explaining relationships between parameters or usage examples. Baseline 3 is appropriate when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Create a new VLAN' clearly states the action (create) and resource (VLAN), which is better than a tautology. However, it lacks specificity about what a VLAN entails in this context and doesn't distinguish it from sibling tools like 'update_vlan' or 'delete_vlan' beyond the basic verb difference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing interface details), when not to use it (e.g., for modifying existing VLANs), or refer to sibling tools like 'update_vlan' or 'delete_vlan' for context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_firewall_ruleC
Delete a firewall rule
| Name | Required | Description | Default |
|---|---|---|---|
| uuid | Yes | Firewall rule UUID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It states the action ('Delete') which implies a destructive operation, but doesn't specify whether deletion is permanent, requires admin permissions, affects network traffic immediately, or has confirmation prompts. For a destructive tool with zero annotation coverage, this leaves critical behavioral traits undocumented.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and resource, making it immediately scannable and understandable without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive operation tool with no annotations and no output schema, the description is incomplete. It doesn't address behavioral aspects like permanence, permissions, or system impact, nor does it explain what happens after deletion (e.g., success confirmation, error conditions). Given the complexity of firewall rule management, more context would be helpful for safe agent operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the single parameter 'uuid' clearly documented as 'Firewall rule UUID'. The description adds no additional parameter information beyond what's in the schema, so it meets the baseline of 3 where the schema does the heavy lifting for parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Delete') and resource ('a firewall rule'), making the purpose immediately understandable. It distinguishes itself from siblings like 'create_firewall_rule' and 'update_firewall_rule' by specifying deletion, though it doesn't explicitly differentiate from 'toggle_firewall_rule' which might also affect rule status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'toggle_firewall_rule' or 'update_firewall_rule'. It doesn't mention prerequisites (e.g., needing the rule UUID from 'find_firewall_rules' or 'list_firewall_rules') or contextual constraints, leaving the agent to infer usage from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_vlanC
Delete a VLAN
| Name | Required | Description | Default |
|---|---|---|---|
| tag | Yes | VLAN tag to delete |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. 'Delete a VLAN' implies a destructive mutation, but it doesn't disclose behavioral traits such as required permissions, whether deletion is reversible, impact on network connectivity, or error handling (e.g., if VLAN doesn't exist). This leaves critical gaps for safe agent operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with a single sentence ('Delete a VLAN'), front-loaded and zero waste. It efficiently conveys the core action without unnecessary words, though this conciseness comes at the cost of completeness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's destructive nature, no annotations, and no output schema, the description is incomplete. It lacks context on safety, prerequisites, outcomes, or error conditions. For a mutation tool with potential network impact, more guidance is needed to ensure correct agent usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with one parameter ('tag') clearly documented in the schema. The description adds no parameter semantics beyond what the schema provides (e.g., no context on tag format or validation). Baseline score of 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Delete a VLAN' clearly states the action (delete) and resource (VLAN), but it's overly basic and doesn't differentiate from sibling tools like 'delete_firewall_rule' or 'haproxy_backend_delete' beyond the resource name. It lacks specificity about what deletion entails (e.g., removing network configuration).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., VLAN must exist), consequences (e.g., devices on VLAN may be affected), or related tools like 'list_vlans' for checking availability or 'create_vlan' for reversal. The description offers only the bare action.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
find_arp_by_hostnameC
Find ARP entries by hostname pattern
| Name | Required | Description | Default |
|---|---|---|---|
| pattern | Yes | Hostname pattern to search |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool finds ARP entries by hostname pattern, implying a read-only search operation, but lacks details on permissions, rate limits, output format, or error handling. For a tool with no annotations, this is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is front-loaded and wastes no space, making it easy to understand at a glance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is incomplete. It doesn't explain what the tool returns (e.g., list of entries, error messages), behavioral aspects like performance or limitations, or how it integrates with sibling tools. For a search tool in a network management context, more context is needed for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the 'pattern' parameter clearly documented as 'Hostname pattern to search'. The description adds minimal value by restating 'hostname pattern' but doesn't provide additional context like pattern syntax (e.g., wildcards) or examples. Baseline score of 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Find') and resource ('ARP entries'), specifying the search criterion ('by hostname pattern'). It distinguishes from siblings like 'find_arp_by_ip' and 'find_arp_by_mac' by indicating the search is based on hostname patterns, not IP or MAC addresses. However, it doesn't explicitly differentiate from 'list_arp_entries', which might list all entries without filtering.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives is provided. The description implies usage for searching ARP entries by hostname pattern, but it doesn't specify scenarios, prerequisites, or exclusions. For example, it doesn't clarify if this is for real-time queries or historical data, or how it compares to 'list_arp_entries' for broader searches.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
find_arp_by_interfaceC
Find ARP entries on specific interface
| Name | Required | Description | Default |
|---|---|---|---|
| interface | Yes | Interface name (e.g., "igc3_vlan6", "lan") |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. 'Find ARP entries' suggests a read-only query, but it doesn't disclose important behavioral aspects: whether this requires specific permissions, what format the results take (e.g., list of entries with what fields), whether it's real-time or cached data, or any rate limits. The description is minimal and lacks operational context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at just 6 words, front-loading the essential information ('Find ARP entries on specific interface') with zero wasted words. Every element earns its place, making it immediately scannable and understandable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is insufficiently complete. For a network diagnostic tool querying ARP tables, users need to know what information is returned (MAC addresses, IPs, timestamps), whether results are filtered or complete, and any system-specific behaviors. The minimal description leaves too many operational questions unanswered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with the single 'interface' parameter fully documented in the schema. The description adds no additional parameter semantics beyond what's already in the schema (interface name examples). With complete schema coverage, the baseline score of 3 is appropriate as the description doesn't enhance parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Find ARP entries') and the target resource ('on specific interface'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from its siblings like 'find_arp_by_hostname', 'find_arp_by_ip', 'find_arp_by_mac', or 'list_arp_entries', which all query ARP data differently.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With multiple ARP-related siblings (find_arp_by_hostname, find_arp_by_ip, find_arp_by_mac, list_arp_entries), there's no indication of when interface-based lookup is preferred over other methods or what distinguishes this from 'list_arp_entries' which might return all entries.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
find_arp_by_ipC
Find ARP entries by IP address or subnet
| Name | Required | Description | Default |
|---|---|---|---|
| ipPattern | Yes | IP address, prefix, or subnet (e.g., "10.0.6", "10.0.6.0/24") |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It states the action ('Find') but doesn't disclose behavioral traits such as whether this is a read-only operation, what permissions are required, if it's cached or real-time, rate limits, or what happens on no matches. The description is minimal and lacks critical operational context for a tool that likely queries network data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and uses clear, direct language. Every word earns its place, making it easy to parse quickly without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is incomplete for a tool that queries network data. It doesn't explain what the return values are (e.g., list of ARP entries, error handling), behavioral aspects like read-only nature, or performance implications. For a tool with potential complexity in network environments, this leaves significant gaps for an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with the parameter 'ipPattern' well-documented in the schema (e.g., accepts IP address, prefix, or subnet). The description adds no additional parameter semantics beyond what's in the schema, such as format examples or edge cases. With high schema coverage, the baseline score of 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Find') and resource ('ARP entries') with the specific mechanism ('by IP address or subnet'). It distinguishes from sibling tools like 'find_arp_by_hostname' and 'list_arp_entries' by specifying the lookup method, though it doesn't explicitly contrast them. The purpose is unambiguous but lacks explicit sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'find_arp_by_hostname', 'find_arp_by_mac', or 'list_arp_entries'. The description implies usage for IP-based ARP lookups but offers no context about prerequisites, performance considerations, or exclusions. This leaves the agent to infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
find_arp_by_macC
Find ARP entries by MAC address
| Name | Required | Description | Default |
|---|---|---|---|
| macPattern | Yes | MAC address or partial MAC (with or without colons) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool's function but doesn't describe what the output looks like (e.g., list of entries, error handling), whether it's read-only or has side effects, performance characteristics, or any constraints like rate limits. For a tool with zero annotation coverage, this is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded with the core functionality and avoids redundancy. Every part of the sentence earns its place by specifying the action, resource, and filter criterion.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what ARP entries are, the return format (e.g., structured data, error messages), or behavioral aspects like whether it's a safe read operation. For a tool with no structured metadata, the description should provide more context to compensate, but it remains minimal.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with 'macPattern' clearly documented as accepting MAC addresses or partial MACs with or without colons. The description adds no additional parameter semantics beyond what's in the schema, such as format examples or search behavior details. With high schema coverage, the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Find') and resource ('ARP entries') with the specific filter criterion ('by MAC address'). It distinguishes from siblings like 'find_arp_by_hostname' and 'find_arp_by_ip' by specifying the MAC-based lookup. However, it doesn't explicitly mention what ARP entries are or the scope of the search.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It doesn't mention sibling tools like 'find_device_by_mac' for broader device searches or 'list_arp_entries' for unfiltered listings. The description implies usage for MAC-based ARP lookups but offers no context on prerequisites, limitations, or comparison to other tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
find_device_by_macC
Find device by MAC address
| Name | Required | Description | Default |
|---|---|---|---|
| mac | Yes | MAC address (with or without colons) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only states the action ('find') without disclosing behavioral traits like whether this is a read-only query, what data is returned, if it requires authentication, or potential errors (e.g., invalid MAC format). For a tool with no annotations, this is insufficient transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It's front-loaded and directly states the tool's purpose, making it easy to parse. This is an example of optimal conciseness for a simple tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is incomplete. It doesn't explain what 'find' entails (e.g., returns device details, status, or location), error conditions, or usage context relative to siblings. For a tool with minimal structured data, the description should provide more operational context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the 'mac' parameter documented as 'MAC address (with or without colons)'. The description adds no additional meaning beyond this, as it merely restates the parameter's purpose. With high schema coverage, the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Find device by MAC address' clearly states the action (find) and resource (device), but it's vague about what 'find' means (retrieve details? locate on network?) and doesn't distinguish from sibling tools like 'find_device_by_name' or 'find_arp_by_mac'. It's functional but lacks specificity about the exact operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. With siblings like 'find_device_by_name' and 'find_arp_by_mac', the description doesn't explain if this is for device lookup versus ARP resolution, or when MAC-based search is preferred over other methods. This leaves the agent without context for tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
find_device_by_nameC
Find devices by hostname pattern
| Name | Required | Description | Default |
|---|---|---|---|
| pattern | Yes | Hostname pattern to search (case-insensitive) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It states the search is case-insensitive (implied from schema), but lacks details on permissions, rate limits, output format, or whether it returns partial matches. For a search tool with zero annotation coverage, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise with a single, front-loaded sentence that directly states the tool's purpose. No wasted words or redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a search tool with no annotations and no output schema, the description is too minimal. It doesn't explain what 'devices' refers to, the return format, or behavioral constraints, leaving significant gaps for an AI agent to use it effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents the 'pattern' parameter. The description adds no additional meaning beyond what's in the schema, meeting the baseline of 3 when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'find' and resource 'devices', specifying the search mechanism 'by hostname pattern'. It distinguishes from some siblings like 'find_device_by_mac' but not from broader search tools like 'get_devices_by_interface'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'find_device_by_mac' or 'find_devices_on_vlan'. The description implies usage for hostname-based searches but lacks explicit context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
find_devices_on_vlanC
Find devices on specific VLAN
| Name | Required | Description | Default |
|---|---|---|---|
| vlanTag | Yes | VLAN tag number (e.g., "6" for DMZ) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('Find') but doesn't describe what 'devices' entails (e.g., network endpoints, hosts), the return format, whether it's a read-only operation, or any limitations like rate limits or authentication needs. The description is minimal and lacks critical behavioral context for a tool that queries network data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste: 'Find devices on specific VLAN'. It is appropriately sized and front-loaded, directly stating the tool's purpose without unnecessary elaboration, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (querying network devices), lack of annotations, and no output schema, the description is incomplete. It doesn't explain what 'devices' means, the return format, or any behavioral traits like safety or performance. For a network discovery tool with no structured support, more context is needed to guide effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with 'vlanTag' documented as 'VLAN tag number (e.g., "6" for DMZ)'. The description adds no additional parameter semantics beyond what the schema provides, such as format details or examples. With high schema coverage, the baseline score of 3 is appropriate, as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Find devices on specific VLAN' clearly states the verb ('Find') and resource ('devices'), specifying the scope ('on specific VLAN'). It distinguishes from some siblings like 'find_device_by_mac' or 'find_device_by_name' by focusing on VLAN-based discovery, but doesn't explicitly differentiate from 'get_devices_by_interface' which might overlap in network context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, such as needing VLANs to be configured, or compare to similar tools like 'get_devices_by_interface' or 'find_device_by_mac'. Usage is implied by the name alone, with no explicit context or exclusions provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
find_firewall_rulesC
Find firewall rules by description
| Name | Required | Description | Default |
|---|---|---|---|
| description | Yes | Description to search for |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states the search functionality but doesn't describe what the tool returns (e.g., a list of matching rules, their details, or error handling), whether it's read-only, or any performance considerations like rate limits. This leaves significant gaps for a tool that likely queries a security system.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core functionality ('Find firewall rules by description') with zero wasted words. It's appropriately sized for a simple search tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what the tool returns (e.g., a list of rule objects, success status), behavioral traits like safety or side effects, or how it differs from sibling tools. For a firewall-related tool with potential security implications, this is inadequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the single parameter 'description' documented as 'Description to search for'. The tool description adds no additional meaning beyond this, such as search semantics (e.g., partial matches, case sensitivity) or examples. Baseline 3 is appropriate since the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('find') and resource ('firewall rules') with a specific search criterion ('by description'), making the purpose unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'list_firewall_rules' or 'get_firewall_rule', which would require a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'list_firewall_rules' (which might list all rules) or 'get_firewall_rule' (which might retrieve a specific rule by ID). The description implies usage for searching by description but doesn't mention exclusions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_arp_statsC
Get ARP table statistics
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. 'Get ARP table statistics' implies a read-only operation but provides no information about authentication requirements, rate limits, response format, or whether this returns aggregated data versus raw entries. The description is too minimal for a tool with no annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at just three words, with zero wasted language. It's front-loaded with the essential action and resource, making it immediately understandable at a basic level. Every word earns its place in this minimal description.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of network management tools and the lack of both annotations and output schema, the description is insufficient. 'Get ARP table statistics' doesn't explain what format the statistics come in, what specific metrics are included, or how this differs from the multiple ARP-related sibling tools. For a tool in this domain with rich sibling alternatives, more context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters with 100% schema description coverage, so the schema fully documents the parameter situation. The description doesn't need to compensate for any parameter gaps. The baseline for this scenario is 4 since the description doesn't add parameter information but doesn't need to given the complete schema coverage for zero parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get ARP table statistics' clearly states the action ('Get') and resource ('ARP table statistics'), providing a basic understanding of what the tool does. However, it doesn't differentiate from sibling tools like 'list_arp_entries' or 'find_arp_by_*' tools, leaving ambiguity about what specifically distinguishes this tool from those alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With multiple ARP-related sibling tools (list_arp_entries, find_arp_by_hostname, find_arp_by_interface, find_arp_by_ip, find_arp_by_mac), the agent receives no indication of whether this tool provides summary statistics versus detailed entries, or what context would make this the appropriate choice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_devices_by_interfaceB
Group devices by network interface
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It states the action ('group') but doesn't disclose behavioral traits: it doesn't specify if this is a read-only operation, what data format is returned (e.g., list, dictionary), whether it requires authentication, or any rate limits. For a tool with zero annotation coverage, this is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence: 'Group devices by network interface.' It's front-loaded with the core action and criterion, with zero wasted words. Every part of the sentence contributes directly to understanding the tool's function.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, no output schema, and zero parameters, the description is minimal. It states what the tool does but lacks completeness: it doesn't explain the return format (e.g., grouped data structure), error conditions, or dependencies. For a tool that likely returns structured data, this leaves the agent with insufficient context to use it effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters, and schema description coverage is 100%, so no parameter documentation is needed. The description doesn't add parameter details, which is appropriate here. Baseline is 4 for zero parameters, as it avoids unnecessary repetition and focuses on the tool's purpose.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Group devices by network interface' clearly states the verb ('group') and resource ('devices'), specifying the grouping criterion ('by network interface'). It distinguishes from siblings like 'find_device_by_mac' or 'get_guest_devices' by focusing on grouping rather than finding or filtering. However, it doesn't explicitly mention what 'devices' refers to (e.g., network devices, endpoints), leaving some ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It doesn't specify prerequisites (e.g., needing prior interface data from 'get_interfaces'), exclusions (e.g., not for real-time monitoring), or compare to siblings like 'find_devices_on_vlan' for VLAN-based grouping. The description implies usage for grouping but offers no contextual boundaries.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_firewall_ruleC
Get firewall rule details
| Name | Required | Description | Default |
|---|---|---|---|
| uuid | Yes | Firewall rule UUID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool 'gets' details, implying a read-only operation, but doesn't clarify if it requires authentication, has rate limits, returns structured data, or handles errors. The description is minimal and misses key behavioral traits needed for safe and effective use in an unannotated context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It's front-loaded with the core action and resource, making it easy to parse. However, it's overly concise to the point of under-specification, lacking necessary details for a tool with no annotations or output schema, which slightly reduces its effectiveness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (a read operation with a required parameter), lack of annotations, and no output schema, the description is incomplete. It doesn't explain what details are returned, error conditions, or behavioral constraints. For a tool that likely returns structured firewall rule data, this leaves significant gaps for an AI agent to operate effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the 'uuid' parameter clearly documented as 'Firewall rule UUID'. The description adds no additional meaning beyond what the schema provides (e.g., no format examples or context about UUID sourcing). With high schema coverage, the baseline score of 3 is appropriate, as the description doesn't compensate but also doesn't detract.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get firewall rule details' clearly states the action (get) and resource (firewall rule), but it's vague about what 'details' includes. It distinguishes from siblings like 'list_firewall_rules' (which likely returns multiple rules) by focusing on a single rule, but doesn't explicitly differentiate from 'find_firewall_rules' (which might search). The purpose is understandable but lacks specificity about the scope of details returned.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a UUID), exclusions, or compare to siblings like 'list_firewall_rules' (for bulk retrieval) or 'find_firewall_rules' (for searching). Usage is implied only by the tool name, leaving the agent to infer context without explicit direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_guest_devicesB
Get all devices on guest network (VLAN 4)
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only states what it does, not how it behaves. It doesn't disclose whether this is a read-only operation, if it requires authentication, rate limits, pagination, return format, or error conditions. 'Get all' suggests a list operation, but behavioral details are missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose. Every word earns its place: 'Get' (action), 'all devices' (resource scope), 'on guest network' (context), '(VLAN 4)' (specific detail). No wasted words or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (0 parameters, no annotations, no output schema), the description is minimally adequate. It states what the tool does but lacks behavioral context (e.g., return format, performance). For a read operation with no parameters, it's complete enough to understand the purpose but not rich in operational details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters with 100% schema description coverage (empty schema). The description adds no parameter information, which is appropriate since there are no parameters. Baseline 4 applies as the description doesn't need to compensate for any parameter gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and resource 'devices on guest network', specifying the scope with 'VLAN 4'. It distinguishes from siblings like 'find_devices_on_vlan' by focusing on a specific VLAN, but doesn't explicitly differentiate from 'get_devices_by_interface' or 'find_device_by_mac' in terms of methodology.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when needing all devices on the guest network (VLAN 4), but provides no explicit guidance on when to use this versus alternatives like 'find_devices_on_vlan' (which might allow VLAN parameterization) or other device lookup tools. No exclusions or prerequisites are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_interfacesB
List available network interfaces
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. 'List available network interfaces' implies a read-only operation, but it doesn't specify whether this requires permissions, what format the output is in, if there are rate limits, or if it's a real-time snapshot. For a tool with zero annotation coverage, this is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It's front-loaded with the core purpose and appropriately sized for a simple tool, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 0 parameters and no output schema, the description is minimally complete for a basic listing operation. However, with no annotations and many sibling networking tools, it lacks context about output format, permissions, or differentiation from alternatives, leaving room for improvement in guiding the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters, and the schema description coverage is 100% (since there are no parameters to describe). The description doesn't need to add parameter details, so it meets the baseline expectation for a parameterless tool without needing to compensate for gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List available network interfaces' clearly states the action (list) and resource (network interfaces). It's specific enough to understand what the tool does, though it doesn't explicitly differentiate from sibling tools like 'get_devices_by_interface' or 'find_arp_by_interface' which might be related but serve different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With many sibling tools related to networking (e.g., 'get_devices_by_interface', 'find_arp_by_interface'), there's no indication of context, prerequisites, or exclusions for using this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_vlanC
Get VLAN details
| Name | Required | Description | Default |
|---|---|---|---|
| tag | Yes | VLAN tag number |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It states 'Get' which implies a read-only operation, but doesn't disclose behavioral traits like whether it requires authentication, what happens if the VLAN doesn't exist (e.g., error handling), or the format of returned details. This leaves significant gaps for a tool with no annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with just three words, which is efficient for a simple tool. However, it's arguably too briefāit could benefit from slightly more context without becoming verbose. The structure is front-loaded but minimal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is incomplete. It doesn't explain what 'details' includes (e.g., configuration, status), error conditions, or return format. For a tool with one required parameter and no structured output documentation, this leaves the agent with insufficient context to use it effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with the single parameter 'tag' documented as 'VLAN tag number'. The description doesn't add any meaning beyond this, such as format constraints (e.g., numeric range) or examples. Baseline 3 is appropriate since the schema already fully describes the parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get VLAN details' clearly states the action (get) and resource (VLAN details), which is better than a tautology. However, it's somewhat vague about what 'details' encompasses and doesn't differentiate from sibling tools like 'list_vlans' or 'find_devices_on_vlan' that also retrieve VLAN-related information.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. With siblings like 'list_vlans' (likely returns all VLANs) and 'find_devices_on_vlan' (returns devices on a VLAN), the description doesn't clarify that this tool retrieves configuration details for a specific VLAN identified by tag.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
haproxy_acl_createC
Create an ACL for HAProxy frontend
| Name | Required | Description | Default |
|---|---|---|---|
| expression | Yes | ACL expression | |
| frontend | Yes | Frontend UUID | |
| name | Yes | ACL name |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only states 'Create' without disclosing behavioral traits. It doesn't mention whether this is a mutating operation, what permissions are required, whether ACLs have specific constraints, what happens on failure, or what the expected response format is.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It's appropriately sized for a tool with good schema coverage and gets straight to the point without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a creation tool with no annotations and no output schema, the description is insufficient. It doesn't explain what happens after creation, whether the ACL becomes active immediately, what validation occurs, or what format the result takes. The context signals show this is a 3-parameter tool with no behavioral guidance.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all three parameters. The description doesn't add any meaning beyond what the schema provides about parameters, nor does it explain relationships between parameters (e.g., that the frontend must exist, ACL naming conventions, or expression syntax).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create') and resource ('an ACL for HAProxy frontend'), providing specific verb+resource pairing. However, it doesn't distinguish this tool from sibling HAProxy tools like 'haproxy_action_create' or 'haproxy_backend_create' beyond the ACL focus.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided about when to use this tool versus alternatives. The description doesn't mention prerequisites, related tools like 'haproxy_frontend_create' that might need to exist first, or when to choose this over other ACL-related operations that might exist in the system.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
haproxy_action_createC
Create an action for HAProxy frontend
| Name | Required | Description | Default |
|---|---|---|---|
| backend | No | Backend name (for use_backend) | |
| condition | No | ACL condition | |
| frontend | Yes | Frontend UUID | |
| type | Yes | Action type | |
| value | No | Action value |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only states it 'creates' without disclosing behavioral traits. It doesn't mention whether this is a write operation (implied but not explicit), what permissions are required, if it's idempotent, or what happens on success/failure, leaving significant gaps for a creation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that directly states the tool's purpose without any fluff. It's appropriately sized and front-loaded, with every word earning its place in conveying the core function.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a creation tool with no annotations and no output schema, the description is incomplete. It doesn't address what the tool returns, error conditions, or side effects, which are critical for an agent to use it correctly in a configuration management context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are well-documented in the schema itself. The description adds no additional parameter semantics beyond the schema, such as explaining relationships between parameters (e.g., how 'type' affects 'value' usage). Baseline 3 is appropriate since the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create an action') and the resource ('for HAProxy frontend'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like haproxy_acl_create or haproxy_backend_create, which also create HAProxy configuration elements, so it misses full sibling distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. The description doesn't mention prerequisites (e.g., needing an existing frontend), exclusions, or comparisons to other HAProxy tools like haproxy_acl_create, leaving the agent with no usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
haproxy_backend_createC
Create a new HAProxy backend
| Name | Required | Description | Default |
|---|---|---|---|
| balance | Yes | Load balancing algorithm | |
| description | No | ||
| mode | Yes | Backend mode | |
| name | Yes | Backend name | |
| servers | No | List of backend servers |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure but provides minimal information. 'Create' implies a write/mutation operation, but there's no mention of permissions required, whether the creation is idempotent, what happens on conflicts, or what the response looks like. For a creation tool with zero annotation coverage, this is inadequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that gets straight to the point with zero wasted words. It's appropriately sized for a tool with good schema documentation and is perfectly front-loaded with the essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a creation tool with no annotations and no output schema, the description is insufficient. It doesn't explain what happens after creation, whether there are side effects, what validation occurs, or how to verify success. The high schema coverage helps with inputs, but the overall context for using this mutation tool is incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 80%, providing good documentation for most parameters. The description adds no additional parameter information beyond what's in the schema. With high schema coverage, the baseline is 3 - the description doesn't compensate but doesn't need to since the schema does most of the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create') and resource ('new HAProxy backend'), making the purpose immediately understandable. It doesn't differentiate from sibling tools like haproxy_backend_delete or haproxy_backend_list, but the verb+resource combination is specific enough for basic understanding.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided about when to use this tool versus alternatives. There's no mention of prerequisites, when this creation should occur in a workflow, or how it relates to sibling tools like haproxy_frontend_create or haproxy_backend_list. The description only states what it does, not when to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
haproxy_backend_deleteC
Delete an HAProxy backend
| Name | Required | Description | Default |
|---|---|---|---|
| uuid | Yes | Backend UUID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'Delete' implies a destructive mutation, the description doesn't specify whether this action is reversible, what permissions are required, whether it affects associated resources (e.g., frontends), or what happens on success/failure. For a destructive tool with zero annotation coverage, this is inadequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that gets straight to the point with zero wasted words. It's appropriately sized for a simple delete operation and front-loads the essential information. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive mutation tool with no annotations and no output schema, the description is insufficient. It doesn't cover behavioral aspects like irreversibility, error conditions, or system impact. Given the complexity of deleting a critical HAProxy component and the lack of structured safety information, the description should provide more context about what 'delete' entails.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with the single parameter 'uuid' clearly documented as 'Backend UUID'. The description doesn't add any meaning beyond what the schema providesāit doesn't explain how to obtain the UUID or what format it expects. With complete schema coverage, the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Delete') and resource ('HAProxy backend'), making the tool's purpose immediately understandable. It distinguishes itself from sibling tools like 'haproxy_backend_create' and 'haproxy_backend_list' by specifying the delete operation. However, it doesn't explicitly mention what constitutes a 'backend' or provide additional context about the resource being deleted.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing the backend UUID from 'haproxy_backend_list'), consequences of deletion, or when not to use it (e.g., if the backend is currently in use). With multiple HAProxy-related siblings available, this lack of differentiation is a significant gap.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
haproxy_backend_healthC
Get health status of a specific backend
| Name | Required | Description | Default |
|---|---|---|---|
| backend | Yes | Backend name |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It states this is a read operation ('Get'), but doesn't describe what the health status includes, format of return data, error conditions, or any side effects. For a monitoring tool with zero annotation coverage, this leaves significant behavioral gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that communicates the core purpose without unnecessary words. It's appropriately sized for a simple read operation and front-loads the essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a monitoring tool with no annotations and no output schema, the description is insufficient. It doesn't explain what health information is returned, format of response data, or how to interpret results. Given the complexity of HAProxy health monitoring and lack of structured output documentation, the description should provide more context about what 'health status' entails.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with the single parameter 'backend' documented as 'Backend name'. The description adds no additional parameter context beyond what's in the schema, so it meets the baseline for high schema coverage but doesn't provide extra value like explaining what constitutes a valid backend name or where to find available backends.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get health status') and target resource ('of a specific backend'), making the purpose immediately understandable. It doesn't distinguish from siblings like haproxy_backend_list or haproxy_stats, but the verb+resource combination is specific enough for basic understanding.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like haproxy_backend_list or haproxy_stats. There's no mention of prerequisites, expected context, or comparison with sibling tools that might provide related HAProxy information.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
haproxy_backend_listB
List all HAProxy backends
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states it's a list operation, implying read-only behavior, but doesn't mention any traits like pagination, rate limits, authentication needs, or output format. This leaves significant gaps for a tool with no annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that directly states the tool's purpose without any fluff. It's front-loaded and wastes no words, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is adequate but minimal. It covers the basic purpose but lacks behavioral context that would be helpful for an agent, such as what the output looks like or any operational constraints.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately doesn't discuss parameters, and since there are none, it meets the baseline expectation without adding unnecessary details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List') and resource ('HAProxy backends'), making the tool's purpose immediately understandable. It doesn't differentiate from sibling tools like 'haproxy_frontend_list' or 'haproxy_certificate_list', but it's specific enough to identify what it does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'haproxy_backend_health' or 'haproxy_backend_create'. The description implies it's for listing backends but doesn't specify context, prerequisites, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
haproxy_certificate_createC
Create a certificate for HAProxy
| Name | Required | Description | Default |
|---|---|---|---|
| ca | No | CA certificate (for import) | |
| certificate | No | Certificate content (for import) | |
| cn | No | Common name (for self-signed) | |
| key | No | Private key (for import) | |
| name | Yes | Certificate name | |
| san | No | Subject alternative names | |
| type | Yes | Certificate type |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure but offers minimal information. It states 'Create' which implies a write/mutation operation, but doesn't describe permissions required, whether the operation is idempotent, what happens on failure, or how the certificate integrates with HAProxy (e.g., if it becomes immediately active). The description lacks crucial behavioral context for a creation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, focused sentence with zero wasted words. It's appropriately sized for a tool with comprehensive schema documentation and gets straight to the point without unnecessary elaboration or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a certificate creation tool with 7 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what the tool returns (e.g., success confirmation, certificate ID), error conditions, or how the created certificate is used in HAProxy. The combination of mutation behavior and missing output information creates significant gaps for agent understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 7 parameters thoroughly with descriptions and an enum for 'type'. The description adds no parameter-specific information beyond what's in the schema, not explaining relationships between parameters (e.g., how 'type' affects which other parameters are relevant) or providing usage examples. This meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create') and resource ('a certificate for HAProxy'), making the purpose unambiguous. It distinguishes from sibling tools like 'haproxy_certificate_list' by specifying creation rather than listing. However, it doesn't explicitly differentiate from other certificate-related tools that might exist in broader contexts.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., when certificates are needed in HAProxy configuration), compare to other certificate management methods, or specify scenarios where creation is appropriate versus listing existing certificates with 'haproxy_certificate_list'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
haproxy_certificate_listB
List available certificates for HAProxy
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states it's a list operation, implying read-only behavior, but doesn't specify output format, pagination, or any constraints like rate limits or authentication needs. This leaves significant gaps for a tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no wasted words. It's front-loaded with the core action and resource, making it efficient and easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is minimally adequate. It states what the tool does but lacks details on behavior, output, or usage context, which could be helpful for an agent despite the low complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters, and schema description coverage is 100%, so no parameter information is needed. The description appropriately doesn't mention parameters, aligning with the schema, which justifies a high baseline score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('List') and resource ('available certificates for HAProxy'), making the purpose specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'haproxy_certificate_create' or other list operations, which prevents a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites, context, or exclusions, leaving the agent to infer usage from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
haproxy_frontend_createC
Create a new HAProxy frontend
| Name | Required | Description | Default |
|---|---|---|---|
| acls | No | Access control lists | |
| backend | Yes | Default backend name | |
| bind | Yes | Bind address (e.g., 0.0.0.0:443) | |
| certificates | No | Certificate UUIDs or names | |
| description | No | ||
| mode | Yes | Frontend mode | |
| name | Yes | Frontend name | |
| ssl | No | Enable SSL |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. 'Create' implies a write/mutation operation, but it doesn't disclose behavioral traits like whether this requires admin permissions, if it's idempotent, what happens on duplicate names, or if changes take effect immediately. For a creation tool with zero annotation coverage, this is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It's appropriately sized and front-loaded, stating the core purpose immediately without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a creation tool with 8 parameters, no annotations, and no output schema, the description is incomplete. It doesn't address what the tool returns (e.g., success confirmation, frontend ID), error conditions, or operational impact. The high schema coverage helps, but the description alone lacks sufficient context for safe and effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is high at 88%, so most parameters are documented in the schema itself. The description adds no additional parameter semantics beyond implying 'new' creation. It doesn't explain relationships between parameters (e.g., how 'ssl' interacts with 'certificates') or provide examples. Baseline 3 is appropriate when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Create a new HAProxy frontend' clearly states the action (create) and resource (HAProxy frontend), which is specific and unambiguous. However, it doesn't differentiate from sibling tools like haproxy_backend_create or haproxy_certificate_create, which would require mentioning what distinguishes a frontend from other HAProxy components.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. There's no mention of prerequisites (e.g., needing a backend first), when not to use it, or how it relates to sibling tools like haproxy_frontend_list or haproxy_frontend_delete. The agent must infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
haproxy_frontend_deleteC
Delete an HAProxy frontend
| Name | Required | Description | Default |
|---|---|---|---|
| uuid | Yes | Frontend UUID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. While 'Delete' implies a destructive mutation, the description doesn't specify whether this action is reversible, what permissions are required, what happens to associated resources, or what the response looks like. For a destructive operation with zero annotation coverage, this is inadequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's appropriately sized for a simple deletion operation and front-loads the essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive deletion tool with no annotations and no output schema, the description is incomplete. It doesn't address critical context like what happens after deletion, error conditions, or confirmation requirements. The combination of a mutation operation with minimal structured data requires more comprehensive description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with the single parameter 'uuid' documented as 'Frontend UUID'. The description adds no additional parameter information beyond what the schema already provides. With high schema coverage, the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete') and resource ('HAProxy frontend'), providing specific verb+resource pairing. However, it doesn't distinguish this tool from sibling deletion tools like haproxy_backend_delete or delete_firewall_rule, which would require explicit differentiation to earn a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. There's no mention of prerequisites (like needing the frontend UUID), when not to use it, or what alternatives exist (such as haproxy_frontend_list to verify before deletion).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
haproxy_frontend_listB
List all HAProxy frontends
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. 'List all HAProxy frontends' implies a read-only operation but doesn't specify details like output format (e.g., JSON array, table), pagination, error handling, or dependencies (e.g., requires HAProxy service running). For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence ('List all HAProxy frontends') with zero wasted words. It's front-loaded with the core action and resource, making it immediately scannable. Every word earns its place, and no structural improvements are needed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is minimally adequate. It states what the tool does but lacks behavioral details (e.g., output format) and usage context. For a read-only list tool, this might suffice, but the absence of output schema means the description should ideally hint at return values, which it doesn't.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters, and schema description coverage is 100%, so no parameter documentation is needed. The description doesn't add parameter semantics, but that's unnecessary here. A baseline of 4 is appropriate since the schema fully covers the absence of parameters, and the description doesn't need to compensate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List all HAProxy frontends' clearly states the verb ('List') and resource ('HAProxy frontends'), making the purpose immediately understandable. It distinguishes from siblings like 'haproxy_frontend_create' and 'haproxy_frontend_delete' by specifying a read-only listing operation. However, it doesn't explicitly differentiate from 'haproxy_backend_list' or other list tools, keeping it from a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., HAProxy configuration), compare to other list tools (e.g., 'haproxy_backend_list'), or specify use cases (e.g., monitoring, troubleshooting). Without any contextual cues, the agent must infer usage from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
haproxy_service_controlC
Control HAProxy service (start, stop, restart, reload)
| Name | Required | Description | Default |
|---|---|---|---|
| action | Yes | Service action to perform |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'control' implies a mutation operation, it doesn't specify whether this requires elevated privileges, what happens on failure (e.g., service state after a failed restart), or if actions like 'stop' are destructive to active connections. The description lacks critical behavioral context for a service control tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (one short phrase) and front-loaded with all necessary core information. Every word earns its placeāthere's no redundancy or unnecessary elaboration. It efficiently communicates the tool's scope in minimal text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of service control (a mutation operation with potential side effects), the description is incomplete. With no annotations and no output schema, it fails to address critical aspects like required permissions, error behavior, or what 'status' returns. For a tool that can stop/restart a critical service, this lack of context is a significant gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with the single parameter 'action' fully documented in the schema (including enum values and description). The description adds no additional parameter semantics beyond what's already in the structured schema, so it meets the baseline of 3 for high schema coverage without adding value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with specific verbs ('start, stop, restart, reload') and identifies the resource ('HAProxy service'). It distinguishes itself from sibling tools that focus on HAProxy configuration (e.g., haproxy_backend_create) by targeting service control operations. However, it doesn't explicitly differentiate from potential non-HAProxy service control tools in the broader set.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., HAProxy must be installed), when to choose 'reload' over 'restart', or how it differs from other service management tools that might exist in the environment. Usage is implied but not explicitly stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
haproxy_statsC
Get HAProxy statistics
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It states 'Get' implies a read operation, but doesn't disclose behavioral traits such as whether it requires authentication, has rate limits, returns real-time or historical data, or if it's safe for frequent use. This leaves significant gaps in understanding how the tool behaves.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words, making it easy to parse. However, it could be slightly improved by front-loading more specific details, but its brevity is appropriate for the simple purpose stated.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of HAProxy systems and no output schema, the description is incomplete. It doesn't explain what statistics are returned (e.g., metrics format, data types) or how to interpret them, which is crucial for an agent to use the tool effectively in a network management context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters, and schema description coverage is 100%, so no parameter documentation is needed. The description doesn't add parameter semantics, but this is acceptable given the lack of parameters, aligning with the baseline for zero parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get HAProxy statistics' clearly states the verb 'Get' and resource 'HAProxy statistics', making the purpose understandable. However, it lacks specificity about what statistics are retrieved (e.g., performance metrics, configuration status) and doesn't distinguish from sibling tools like haproxy_backend_list or haproxy_frontend_list, which might provide overlapping or related information.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. With siblings like haproxy_backend_list and haproxy_frontend_list, the description doesn't clarify if this tool aggregates broader statistics or serves a different purpose, leaving the agent to guess based on context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
iac_apply_deploymentC
Apply a deployment plan
| Name | Required | Description | Default |
|---|---|---|---|
| autoApprove | No | Skip confirmation | |
| planId | Yes | Plan ID from plan_deployment |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden but lacks behavioral details. 'Apply' implies a mutation, but it doesn't disclose if it's destructive, requires specific permissions, has side effects, or involves rate limits. This is inadequate for a tool that likely modifies infrastructure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It's front-loaded and appropriately sized for the tool's complexity, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, no output schema, and a likely complex mutation operation (infrastructure deployment), the description is incomplete. It doesn't cover behavioral traits, return values, or error conditions, leaving significant gaps for an agent to understand the tool's impact.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters ('planId' and 'autoApprove'). The description adds no meaning beyond this, such as explaining what 'apply' does with the plan or the implications of 'autoApprove'. Baseline 3 is appropriate as the schema handles parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Apply a deployment plan' states the action (apply) and target (deployment plan), but it's vague about what 'apply' entailsāwhether it executes, validates, or deploys infrastructure changes. It doesn't differentiate from sibling tools like 'iac_plan_deployment' or 'iac_destroy_deployment', leaving ambiguity in scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., requiring a plan from 'iac_plan_deployment'), exclusions, or comparisons to siblings like 'iac_destroy_deployment', leaving the agent without context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
iac_destroy_deploymentC
Destroy deployed resources
| Name | Required | Description | Default |
|---|---|---|---|
| deploymentId | Yes | Deployment to destroy | |
| force | No | Force destruction |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. 'Destroy' implies a destructive, irreversible operation, but the description doesn't disclose critical behavioral traits like whether destruction is immediate, requires confirmation, affects dependencies, or has side effects. The 'force' parameter hints at optional safeguards, but this isn't explained in the description.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with zero wasted words. It directly states the tool's purpose without unnecessary elaboration, making it highly efficient for quick comprehension.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive tool with no annotations and no output schema, the description is inadequate. It lacks context on what 'destroy' entails (e.g., permanent deletion, cleanup processes), expected outcomes, error conditions, or integration with sibling tools like 'iac_apply_deployment'. This leaves significant gaps for safe agent usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are documented in the schema. The description adds no meaning beyond the schemaāit doesn't explain what a 'deploymentId' refers to, how to obtain it, or the implications of the 'force' flag. Baseline 3 is appropriate as the schema handles parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Destroy deployed resources' clearly states the action (destroy) and target (deployed resources), which is specific and unambiguous. However, it doesn't differentiate from sibling tools like 'delete_firewall_rule' or 'delete_vlan' by specifying what type of resources (e.g., infrastructure deployments) are affected.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a deployment ID from 'iac_list_resource_types' or 'iac_plan_deployment'), nor does it warn against misuse (e.g., irreversible destruction).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
iac_list_resource_typesB
List available resource types
| Name | Required | Description | Default |
|---|---|---|---|
| category | No | Filter by category (network, firewall, services) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'List' implies a read-only operation, the description doesn't address critical aspects like whether this tool requires authentication, returns paginated results, has rate limits, or what format the output takes. For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core purpose and avoids unnecessary elaboration. This makes it easy for an agent to parse quickly while still conveying essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (one optional parameter) and high schema coverage, the description is minimally adequate. However, with no annotations and no output schema, it fails to address behavioral aspects like authentication needs or return format. For a simple list tool, this is acceptable but leaves room for improvement in guiding the agent fully.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the single parameter 'category' clearly documented as filtering by categories like 'network, firewall, services'. The description adds no additional parameter information beyond what the schema provides. According to scoring rules, when schema coverage is high (>80%), the baseline score is 3 even without parameter details in the description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List available resource types' clearly states the verb ('List') and resource ('resource types'), making the tool's purpose immediately understandable. It distinguishes itself from siblings like 'list_firewall_rules' or 'list_vlans' by focusing on resource types rather than specific resources. However, it doesn't specify what 'resource types' refers to in the context of infrastructure-as-code (IaC), leaving some ambiguity about the exact scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, typical use cases, or how it relates to sibling tools like 'iac_apply_deployment' or 'iac_plan_deployment'. The agent must infer usage from the tool name and context alone, which is insufficient for optimal decision-making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
iac_plan_deploymentC
Plan infrastructure deployment changes
| Name | Required | Description | Default |
|---|---|---|---|
| dryRun | No | Preview changes without applying | |
| name | Yes | Deployment name | |
| resources | Yes | Resources to deploy |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but offers minimal behavioral insight. 'Plan' implies a read-only preview operation, but it doesn't disclose whether this requires specific permissions, what the planning output looks like, whether it validates configurations, or if it has side effects. The dryRun parameter suggests preview behavior, but this isn't explained in the description itself.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient phrase that gets straight to the point without unnecessary words. It's appropriately sized for a tool with good schema documentation, though it could be more informative given the lack of annotations and output schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 3 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what 'planning' entails operationally, what format the plan output takes, or how this tool fits into a deployment workflow. The context signals indicate this is a moderately complex tool that needs more descriptive context than provided.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are well-documented in the schema. The description adds no additional parameter context beyond what's already in the schema descriptions. The baseline of 3 is appropriate since the schema does the heavy lifting, though the description doesn't compensate with any extra semantic meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Plan infrastructure deployment changes' states a general purpose (planning changes) but lacks specificity about what 'infrastructure deployment' entails or what kind of changes are planned. It distinguishes from obvious siblings like 'iac_apply_deployment' and 'iac_destroy_deployment' by the 'plan' verb, but doesn't clarify what differentiates it from other planning or analysis tools in the sibling list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. While the name suggests it's for planning before applying deployments, the description doesn't explicitly state this relationship with 'iac_apply_deployment' or mention prerequisites, timing considerations, or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_arp_entriesB
List all ARP table entries
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It states it's a list operation, implying read-only behavior, but doesn't specify output format, pagination, rate limits, permissions required, or whether it returns live data vs cached entries. For a tool with zero annotation coverage, this leaves significant behavioral gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and resource, making it immediately scannable and appropriately sized for a simple listing tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters, no annotations, and no output schema, the description provides the minimum viable information about what the tool does. However, for a tool that likely returns structured network data, more context about output format or behavioral characteristics would be helpful, especially with many similar ARP tools available.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters with 100% schema description coverage (empty schema). The description doesn't need to explain parameters, and 'all' implies no filtering parameters are needed. This meets expectations for a parameterless tool, though it could theoretically mention if certain implicit filters apply.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List all ARP table entries' clearly states the verb ('List') and resource ('ARP table entries'), making the tool's purpose immediately understandable. It distinguishes from siblings like 'find_arp_by_ip' or 'get_arp_stats' by specifying it returns all entries without filtering. However, it doesn't explicitly contrast with these siblings in the description text itself.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'find_arp_by_ip' or 'get_arp_stats'. It doesn't mention prerequisites, context, or exclusions, leaving the agent to infer usage from the tool name alone among many similar ARP-related tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_backupsB
List available backups
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. 'List available backups' implies a read-only operation but doesn't specify permissions needed, output format, pagination, or error conditions. For a tool with zero annotation coverage, this is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description 'List available backups' is a single, efficient sentence that front-loads the core purpose without any wasted words. It's appropriately sized for a simple tool with no parameters.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, no output schema, and a simple zero-parameter tool, the description is incomplete. It doesn't explain what 'available' means (e.g., time range, status), the return format, or how backups are identified, leaving the agent with insufficient context for reliable use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters with 100% schema description coverage, so the schema fully documents the lack of inputs. The description doesn't add parameter details, but with no parameters to explain, a baseline score of 4 is appropriate as there's nothing missing.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List available backups' clearly states the action (list) and resource (backups), making the tool's purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'create_backup' or 'restore_backup' beyond the obvious verb difference, missing an opportunity to clarify scope or format.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With siblings like 'create_backup' and 'restore_backup', it doesn't specify if this is for inventory checks, pre-restore verification, or other contexts, leaving usage entirely implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_dhcp_leasesC
List all DHCP leases
| Name | Required | Description | Default |
|---|---|---|---|
| interface | No | Filter by interface (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full responsibility for behavioral disclosure. 'List all DHCP leases' implies a read-only operation, but it doesn't specify whether this requires authentication, what format the output takes, if there are rate limits, or how many results are returned. For a network tool with zero annotation coverage, this leaves significant gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It's front-loaded with the core purpose ('List all DHCP leases'), making it immediately clear what the tool does.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations and output schema, the description is incomplete for a network tool. It doesn't explain what DHCP leases are, what data is returned, or any behavioral aspects like permissions or limitations. For a tool in a complex networking context with many siblings, this minimal description leaves too much undefined.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, with the single parameter 'interface' documented as an optional filter. The description mentions 'all DHCP leases' but doesn't add meaning beyond what the schema provides about filtering. With high schema coverage, the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List') and resource ('DHCP leases'), making the purpose immediately understandable. However, it doesn't differentiate this tool from other list tools in the sibling set (like list_arp_entries, list_backups, list_vlans), which would require specifying what makes DHCP leases distinct.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With many sibling tools available (including other list operations and network-related tools), there's no indication of context, prerequisites, or comparisons to help an agent choose appropriately.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_dns_blocklistB
List all DNS blocklist entries
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool lists entries but doesn't describe the return format (e.g., list structure, fields), pagination behavior, or any constraints (e.g., rate limits, permissions required). For a tool with zero annotation coverage, this is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose with no wasted words. It's appropriately sized for a simple list operation with no parameters, making it easy for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is minimally adequate. It states what the tool does but lacks details on behavior, output, or usage context. For a tool with no structured data to rely on, it should provide more guidance to be fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters, and schema description coverage is 100%, so there are no parameters to document. The description doesn't need to add parameter semantics, and it correctly implies no inputs are required. A baseline of 4 is appropriate since the schema fully covers the absence of parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List') and resource ('DNS blocklist entries'), making the purpose immediately understandable. It doesn't specifically differentiate from sibling tools like 'search_dns_blocklist' or 'toggle_blocklist_entry', which prevents a perfect score, but it's unambiguous about what the tool does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'search_dns_blocklist' or 'toggle_blocklist_entry'. It lacks any context about prerequisites, typical use cases, or exclusions, leaving the agent to infer usage based on the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_firewall_rulesC
List all firewall rules
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only states the action ('List all firewall rules') without disclosing behavioral traits such as pagination, rate limits, authentication needs, or output format. This is inadequate for a tool with no annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It's appropriately sized and front-loaded, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, no output schema, and a simple purpose, the description is incomplete. It doesn't address behavioral aspects like output format or constraints, which are crucial for an agent to use the tool effectively in this context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters with 100% schema description coverage, so no parameter information is needed. The description doesn't add parameter semantics, but this is acceptable given the lack of parameters, aligning with the baseline for 0 params.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List all firewall rules' clearly states the verb ('List') and resource ('firewall rules'), but it's vague about scope and lacks sibling differentiation. It doesn't specify whether this lists all rules globally or for a specific context, nor how it differs from 'find_firewall_rules' among siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. With siblings like 'find_firewall_rules' and 'get_firewall_rule', the description doesn't indicate if this is for bulk listing, filtered searches, or specific retrieval, leaving the agent without usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_vlansB
List all VLANs
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. 'List all VLANs' implies a read-only operation but doesn't disclose behavioral traits like pagination, rate limits, authentication needs, or what 'all' entails (e.g., active only, includes system VLANs). For a tool with zero annotation coverage, this is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description 'List all VLANs' is a single, efficient sentence with zero waste. It's front-loaded and appropriately sized for a simple list operation, earning full marks for conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 0 parameters and no output schema, the description is minimal but adequate for a basic list tool. However, with no annotations and no output details, it lacks completeness regarding behavioral context (e.g., return format, error handling). It meets the minimum viable threshold but has clear gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters, and schema description coverage is 100%, so no parameter documentation is needed. The description doesn't add parameter semantics, but with no parameters, a baseline of 4 is appropriate as it doesn't mislead or omit required information.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List all VLANs' clearly states the action (list) and resource (VLANs), making the purpose immediately understandable. It distinguishes from siblings like get_vlan (singular) and create_vlan/delete_vlan (mutations). However, it doesn't specify scope or format, keeping it from a perfect 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like get_vlan (for a specific VLAN) or find_devices_on_vlan (for VLAN details). It lacks context about prerequisites, such as whether VLANs must be configured first, or any exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
macro_analyzeC
Analyze a macro to detect patterns and parameters
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Macro ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions 'detect patterns and parameters,' which hints at a read-only analysis function, but doesn't specify if it's safe, what the output looks like, or any side effects (e.g., whether it modifies the macro). This leaves gaps in understanding the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence ('Analyze a macro to detect patterns and parameters') that is front-loaded and wastes no words. However, it could be more structured by including context or usage hints, but it's appropriately concise for its purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (analysis tool with no annotations and no output schema), the description is incomplete. It doesn't explain what 'patterns and parameters' are detected, the format or nature of the analysis results, or any behavioral traits. For a tool that likely returns insights, more detail is needed to be fully helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with one parameter ('id') clearly documented as 'Macro ID.' The description doesn't add any meaning beyond this, such as format examples or constraints. With high schema coverage, the baseline score of 3 is appropriate, as the schema handles the parameter documentation adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the tool's purpose ('Analyze a macro to detect patterns and parameters'), which is clear but vague. It specifies the verb ('analyze') and resource ('macro'), but doesn't distinguish it from sibling tools like macro_list, macro_play, or macro_generate_tool, nor does it explain what 'patterns and parameters' means in this context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing macro), exclusions, or comparisons to siblings like macro_list (which might list macros) or macro_play (which might execute them). Usage is implied but not explicitly stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
macro_deleteC
Delete a saved macro
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Macro ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Delete' implies a destructive, irreversible mutation, but the description doesn't specify whether deletion is permanent, requires confirmation, affects related resources, or has permission requirements. For a destructive tool with zero annotation coverage, this lack of behavioral details is a significant gap that could lead to unsafe usage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, direct sentence with zero wasted wordsā'Delete a saved macro' efficiently conveys the core action and target. It's appropriately front-loaded and avoids unnecessary elaboration, making it easy to parse quickly. This is an excellent example of conciseness for a simple tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's destructive nature and lack of annotations or output schema, the description is incomplete. It doesn't address critical context like irreversible effects, error handling, or what happens post-deletion (e.g., success confirmation or side effects). For a mutation tool with no structured safety cues, this leaves too much unspecified for reliable agent use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage (the 'id' parameter is documented as 'Macro ID'), so the baseline score is 3. The description doesn't add any parameter semantics beyond what the schema providesāit doesn't explain how to obtain the ID, its format, or validation rules. This meets the minimum viable level given the schema's completeness.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Delete a saved macro' clearly states the action (delete) and target resource (saved macro), making the purpose immediately understandable. It distinguishes from sibling tools like macro_list, macro_play, and macro_export by specifying deletion rather than listing, executing, or exporting. However, it doesn't specify what a 'macro' is in this context or differentiate from other deletion tools like delete_firewall_rule or delete_vlan beyond the resource type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a macro ID from macro_list), exclusions (e.g., not for active macros), or comparisons to similar tools (e.g., macro_export for preservation before deletion). Without such context, users must infer usage from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
macro_exportC
Export all macros to a file
| Name | Required | Description | Default |
|---|---|---|---|
| path | Yes | Export file path |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but offers minimal behavioral insight. It states the action but doesn't disclose critical traits: whether it overwrites existing files, requires specific permissions, handles errors, supports formats (e.g., JSON, CSV), or includes metadata. For a write operation with zero annotation coverage, this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and resource, making it immediately understandable. Every word earns its place without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool that performs a write operation (exporting to a file) with no annotations and no output schema, the description is incomplete. It lacks details on behavior (e.g., file overwriting, error handling), output format, or success indicators. Given the complexity and absence of structured data, more context is needed for safe and effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with the single parameter 'path' documented as 'Export file path'. The description adds no additional meaning beyond this, such as path format requirements (e.g., absolute vs. relative) or file naming conventions. Baseline 3 is appropriate when the schema adequately covers parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Export') and resource ('all macros'), specifying the output format ('to a file'). It distinguishes from siblings like macro_list (which lists rather than exports) and macro_import (which imports rather than exports). However, it doesn't explicitly differentiate from macro_analyze or macro_generate_tool, which might also involve macro processing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing existing macros), compare to macro_list for viewing without export, or specify use cases like backup or migration. The agent must infer usage from context alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
macro_generate_toolC
Generate an MCP tool definition from a macro
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Macro ID | |
| save | No | Save the generated tool to a file |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It states the action ('generate') but doesn't disclose behavioral traits such as whether this is a read-only or mutating operation, what permissions are required, how the output is formatted, or if there are rate limits. For a tool that likely creates definitions, this is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with zero waste. It's front-loaded with the core purpose and appropriately sized for the tool's complexity, making it easy for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (generating definitions likely involves output formatting) and lack of annotations and output schema, the description is incomplete. It doesn't explain what the generated tool definition looks like, how it's returned, or any side effects. For a tool with no structured output documentation, this leaves critical gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters (id and save). The description doesn't add any meaning beyond the schema, such as explaining what a 'macro' is or how the save option affects file output. With high schema coverage, the baseline is 3, as the description doesn't compensate but doesn't detract either.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Generate an MCP tool definition from a macro'. It specifies the verb ('generate') and resource ('MCP tool definition'), and distinguishes it from sibling tools like macro_analyze, macro_play, or macro_list. However, it doesn't explicitly differentiate from macro_export or macro_import, which might also involve tool definitions, so it's not a perfect 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing macro), compare it to sibling tools like macro_export or macro_play, or specify scenarios where generation is appropriate. This leaves the agent without contextual usage cues.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
macro_importC
Import macros from a file
| Name | Required | Description | Default |
|---|---|---|---|
| overwrite | No | Overwrite existing macros | |
| path | Yes | Import file path |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. 'Import' implies a write operation, but the description doesn't disclose critical behavioral traits: whether it requires admin permissions, what happens on success/failure, if it validates file format, or if it's idempotent. The overwrite parameter hints at potential data loss, but this isn't explained in the description.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and resource. No structural issues existāit's appropriately sized for a simple import tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no annotations and no output schema, the description is incomplete. It doesn't address behavioral risks (e.g., data overwriting), expected outcomes, error conditions, or relationship to other macro tools. The agent lacks sufficient context to use this tool safely and effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are fully documented in the schema. The description adds no additional meaning beyond implying a file import action. It doesn't explain parameter interactions (e.g., overwrite behavior) or provide context beyond what's in the schema. Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Import macros from a file' clearly states the action (import) and resource (macros from a file). It distinguishes from siblings like macro_export, macro_list, or macro_delete by specifying the import operation. However, it doesn't explicitly differentiate from macro_analyze or macro_generate_tool, which have different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a valid file), when not to use it, or how it relates to sibling tools like macro_export or macro_play. The agent must infer usage from the name and context alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
macro_listB
List all saved macros
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'List all saved macros,' which implies a read-only operation, but doesn't clarify critical aspects like whether it returns a simple list or detailed metadata, if there are pagination or rate limits, or any authentication requirements. For a tool with zero annotation coverage, this leaves significant behavioral gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence: 'List all saved macros.' It is front-loaded with the core action and resource, with no wasted words or redundant information. This makes it easy to parse and understand quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is minimally adequate. It states what the tool does but lacks details on output format, behavioral constraints, or usage context. For a read-only listing tool, this is a basic but incomplete description that leaves the agent to infer missing information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters, and schema description coverage is 100% (since there are no parameters to describe). The description doesn't need to add parameter semantics, as there are none. A baseline score of 4 is appropriate because the description accurately reflects the lack of inputs, though it doesn't explicitly state 'no parameters required,' which would be a minor enhancement.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List all saved macros' clearly states the verb ('List') and resource ('saved macros'), making the purpose immediately understandable. It distinguishes itself from sibling tools like macro_analyze, macro_delete, macro_play, etc., which perform different operations on macros. However, it doesn't specify scope or format details that would make it fully specific (e.g., 'List all saved macros with their names and descriptions').
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, context for usage, or differentiate from similar listing tools (e.g., list_backups, list_firewall_rules). Without such guidance, an agent must infer usage from the tool name alone, which is insufficient for optimal selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
macro_playC
Play a saved macro
| Name | Required | Description | Default |
|---|---|---|---|
| dryRun | No | Execute in dry-run mode without making actual API calls | |
| id | Yes | Macro ID | |
| parameters | No | Parameters to substitute in the macro |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. 'Play a saved macro' implies execution of recorded actions, but it doesn't disclose whether this is destructive, requires specific permissions, has rate limits, or what happens during execution. The description lacks essential behavioral context for a tool that likely performs operations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's appropriately sized for a tool with clear purpose and good schema documentation, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no annotations and no output schema that likely executes complex operations (given the 'parameters' object and 'dryRun' option), the description is insufficient. It doesn't explain what 'playing' entails, what types of operations might be executed, or what the expected outcome is, leaving significant gaps in understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, so parameters are well-documented in the schema itself. The description adds no additional parameter semantics beyond what's already in the schema, which meets the baseline expectation when schema coverage is complete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Play a saved macro' clearly states the action (play) and resource (saved macro), making the tool's purpose immediately understandable. However, it doesn't differentiate this from sibling tools like 'macro_analyze' or 'macro_list', which prevents a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'macro_analyze' or 'macro_list', nor does it mention prerequisites such as needing a recorded macro first. It simply states what the tool does without contextual usage information.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
macro_start_recordingC
Start recording API calls to create a macro
| Name | Required | Description | Default |
|---|---|---|---|
| description | Yes | Macro description | |
| name | Yes | Macro name |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action but doesn't explain what happens during recording (e.g., which API calls are captured, if there are time limits, or how to stop it). For a tool that initiates a stateful process, this is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded with the core action, making it easy to parse and understand quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of initiating a macro recording process, no annotations, and no output schema, the description is insufficient. It doesn't explain what the tool returns, how to interact with the recording once started, or any behavioral nuances, leaving critical gaps for the agent to operate effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, clearly documenting the 'name' and 'description' parameters. The tool description doesn't add any extra semantic details about these parameters, such as formatting constraints or usage context, so it meets the baseline of 3 where the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Start recording API calls') and the purpose ('to create a macro'), making the tool's function evident. However, it doesn't explicitly differentiate from its sibling 'macro_stop_recording' beyond the obvious start/stop distinction, which is why it doesn't reach a score of 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'macro_play' or 'macro_import', nor does it mention prerequisites or context for starting a recording. It lacks explicit usage instructions, leaving the agent to infer timing from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
macro_stop_recordingB
Stop recording and save the macro
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only states the basic action without disclosing behavioral traits. It doesn't mention whether this operation is destructive, requires specific permissions, what happens to the recorded macro, or any side effects like file creation or system changes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that directly states the tool's purpose without any wasted words. It's appropriately sized for a simple tool and front-loaded with the essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter tool with no output schema, the description is minimally complete but lacks context about the macro system workflow. It doesn't explain what happens after saving (e.g., where the macro is stored, format, or how to use it later), leaving gaps in understanding the full operational context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters with 100% schema description coverage, so the baseline is 4. The description appropriately doesn't discuss parameters since none exist, avoiding redundancy while matching the schema's completeness.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('stop recording and save') and the resource ('the macro'), providing a specific verb+resource combination. However, it doesn't explicitly differentiate from its sibling 'macro_start_recording' beyond the obvious opposite action, which prevents a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, prerequisites (e.g., must have started recording first), or context for its application. It lacks any mention of related tools like 'macro_start_recording' or 'macro_play' for workflow understanding.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
restore_backupC
Restore a configuration backup
| Name | Required | Description | Default |
|---|---|---|---|
| backupId | Yes | Backup ID to restore |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. 'Restore' implies a write operation that could be destructive, but the description doesn't warn about overwriting current configurations, requiring specific permissions, or potential system downtime. It also omits details on response format, error conditions, or idempotency, leaving critical behavioral traits unspecified.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and resource, making it immediately scannable. Every word earns its place by directly contributing to understanding the tool's purpose without redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a restoration operation (potentially destructive, system-altering) and the lack of annotations and output schema, the description is incomplete. It doesn't address critical context like what 'configuration' entails, safety warnings, success/error responses, or dependencies on other tools (e.g., 'list_backups'). For a mutation tool with no structured support, more descriptive detail is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with 'backupId' clearly documented as 'Backup ID to restore'. The description adds no additional parameter semantics beyond what the schema provides, such as format examples or where to obtain backup IDs. Since the schema does the heavy lifting, the baseline score of 3 is appropriate, though the description doesn't compensate for any gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Restore') and the resource ('a configuration backup'), making the tool's purpose immediately understandable. It distinguishes from sibling tools like 'create_backup' and 'list_backups' by focusing on restoration rather than creation or listing. However, it doesn't specify what exactly gets restored (e.g., system settings, network configurations), which prevents a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing backup from 'create_backup' or 'list_backups'), potential side effects, or when not to use it (e.g., during active operations). Without such context, the agent must infer usage from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
search_dns_blocklistC
Search DNS blocklist entries
| Name | Required | Description | Default |
|---|---|---|---|
| pattern | Yes | Pattern to search for in domains or descriptions |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Search DNS blocklist entries' implies a read-only operation but doesn't specify whether it requires authentication, returns paginated results, has rate limits, or what format the results take. For a search tool with zero annotation coverage, this leaves significant behavioral gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and resource, making it immediately understandable without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is incomplete for a search tool. It doesn't explain what the search returns (e.g., matched entries, counts, or error handling), nor does it provide context about search behavior (e.g., case sensitivity, wildcard support). For a tool with one parameter but rich operational context needed, this is inadequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with the single parameter 'pattern' clearly documented as 'Pattern to search for in domains or descriptions'. The description adds no additional parameter semantics beyond what the schema provides, so it meets the baseline of 3 for adequate coverage when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Search DNS blocklist entries' clearly states the verb (search) and resource (DNS blocklist entries), making the purpose understandable. However, it doesn't differentiate from sibling tools like 'list_dns_blocklist' or 'toggle_blocklist_entry', leaving ambiguity about when to use this specific search tool versus other blocklist-related tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With siblings like 'list_dns_blocklist' (which presumably lists all entries) and 'toggle_blocklist_entry' (which modifies entries), there's no indication whether this tool is for filtered searches, partial matches, or specific use cases. The agent must infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
test_connectionB
Test API connection and authentication
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool tests connection and authentication, implying a read-only, non-destructive operation, but doesn't specify what 'test' entails (e.g., returns success/failure, error details, latency metrics) or any side effects (e.g., logging, rate limits). For a tool with zero annotation coverage, this leaves key behavioral aspects unclear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It's front-loaded with the core purpose and appropriately sized for a simple tool. Every part of the description ('Test API connection and authentication') earns its place by conveying essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is adequate but has gaps. It covers the basic purpose but lacks details on usage context, behavioral output, or error handling. For a connection-testing tool in a server with many configuration siblings, more completeness would help the agent understand when and how to invoke it effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters, and schema description coverage is 100% (though empty). The description doesn't need to explain parameters, so it meets the baseline. No additional parameter information is required or provided, which is appropriate for a parameterless tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Test API connection and authentication' clearly states the tool's purpose with a specific verb ('Test') and target ('API connection and authentication'). It distinguishes this from sibling tools that perform configuration, management, or query operations. However, it doesn't explicitly differentiate from potential alternative connection-testing tools (though none are present in the sibling list).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., use after configuration), typical scenarios (e.g., verify setup before other operations), or exclusions. With many sibling tools for various operations, the lack of contextual guidance is a significant gap.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
toggle_blocklist_entryB
Enable/disable a DNS blocklist entry
| Name | Required | Description | Default |
|---|---|---|---|
| uuid | Yes | UUID of the blocklist entry |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. 'Enable/disable' implies a state mutation, but the description doesn't specify whether this requires admin permissions, what happens to dependent systems, whether the change is immediate or requires restart, or what the expected response format is. For a mutation tool with zero annotation coverage, this leaves significant behavioral gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that directly states the tool's purpose with zero wasted words. It's appropriately sized for a simple toggle operation and front-loads the essential information without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter mutation tool with no annotations and no output schema, the description provides the basic purpose but lacks important context. It doesn't explain what 'enable/disable' means operationally, what systems are affected, error conditions, or return values. While concise, it leaves the agent with significant uncertainty about the tool's behavior and effects.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with the single 'uuid' parameter fully documented in the schema. The description adds no parameter-specific information beyond what's already in the schema (which states 'UUID of the blocklist entry'). With high schema coverage, the baseline score of 3 is appropriate as the description doesn't need to compensate for schema gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Enable/disable') and the resource ('a DNS blocklist entry'), providing a specific verb+resource combination. It distinguishes from obvious siblings like 'block_domain' or 'unblock_domain' by focusing on toggling rather than creating/removing entries. However, it doesn't explicitly differentiate from 'toggle_firewall_rule' which has a similar naming pattern but different domain.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing blocklist entry), when-not scenarios, or direct alternatives like using separate enable/disable tools if they existed. The agent must infer usage from the name and context alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
toggle_firewall_ruleC
Toggle firewall rule enabled/disabled
| Name | Required | Description | Default |
|---|---|---|---|
| uuid | Yes | Firewall rule UUID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but offers minimal behavioral insight. It implies a mutation (toggle) but doesn't disclose critical details: whether this requires admin permissions, if changes are immediate or require a restart, potential side effects on network traffic, or error handling. For a security-related mutation tool, this lack of transparency is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and resource, making it immediately understandable. Every word earns its place by conveying essential information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no annotations and no output schema, the description is incomplete. It lacks context on permissions, side effects, return values, or error conditions. Given the complexity of firewall operations and the absence of structured safety hints, more behavioral disclosure is needed to adequately guide an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with the single parameter 'uuid' clearly documented in the schema as 'Firewall rule UUID'. The description adds no additional parameter semantics beyond what the schema provides, such as format examples or where to obtain the UUID. Baseline 3 is appropriate given the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('toggle') and resource ('firewall rule'), specifying it changes between enabled/disabled states. It distinguishes from siblings like 'create_firewall_rule', 'delete_firewall_rule', and 'update_firewall_rule' by focusing on state change rather than creation, deletion, or modification of rule properties.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing rule UUID), when not to use it, or how it differs from similar tools like 'update_firewall_rule' which might also affect rule states. The description only states what it does, not when to apply it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
unblock_domainB
Remove a domain from the DNS blocklist
| Name | Required | Description | Default |
|---|---|---|---|
| domain | Yes | Domain to unblock |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('Remove') but doesn't clarify whether this requires admin permissions, if the change is immediate or requires a restart, what happens if the domain isn't blocked, or if there are rate limits. For a mutation tool with zero annotation coverage, this leaves significant gaps in understanding the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and resource, making it immediately scannable and understandable. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter mutation tool with no annotations and no output schema, the description is minimally adequate. It states what the tool does but lacks behavioral context (permissions, side effects) and usage guidance. The high schema coverage helps, but the description doesn't fully compensate for the missing annotations and output information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the 'domain' parameter clearly documented as 'Domain to unblock'. The description adds no additional parameter details beyond what the schema provides, such as format examples (e.g., 'example.com') or constraints. With high schema coverage, the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Remove') and target resource ('domain from the DNS blocklist'), making the tool's purpose immediately understandable. It doesn't explicitly differentiate from sibling tools like 'toggle_blocklist_entry' or 'apply_blocklist_category', but the verb 'Remove' suggests a specific unblocking action rather than toggling or applying categories.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'toggle_blocklist_entry' or 'block_domain'. It doesn't mention prerequisites (e.g., the domain must be currently blocked), exclusions, or typical use cases. The agent must infer usage from the name and description alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_firewall_ruleC
Update a firewall rule
| Name | Required | Description | Default |
|---|---|---|---|
| description | No | New description | |
| destination | No | New destination | |
| destinationPort | No | New destination port | |
| enabled | No | Enable/disable rule | |
| source | No | New source | |
| sourcePort | No | New source port | |
| uuid | Yes | Firewall rule UUID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only states the action ('update') without disclosing behavioral traits like required permissions, whether changes are immediate or require a restart, potential side effects, or error handling. It lacks details on what 'update' entails beyond the basic operation, leaving significant gaps in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, direct sentence with zero wasted words, making it highly concise and front-loaded. It efficiently states the core action without unnecessary elaboration, though this brevity contributes to gaps in other dimensions.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with 7 parameters, no annotations, and no output schema, the description is inadequate. It lacks context on behavior, usage, and output, failing to compensate for the absence of structured data. The agent would struggle to use this tool effectively without additional information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, clearly documenting all 7 parameters (e.g., 'uuid' for identification, 'enabled' for toggling). The description adds no additional meaning beyond the schema, such as explaining parameter interactions or constraints. Baseline 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Update a firewall rule' restates the tool name with minimal elaboration, making it tautological. It specifies the verb ('update') and resource ('firewall rule') but lacks detail on scope or differentiation from sibling tools like 'toggle_firewall_rule' or 'create_firewall_rule', leaving the purpose vague beyond the obvious.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as 'toggle_firewall_rule' (for enabling/disabling) or 'create_firewall_rule' (for new rules). The description offers no context, prerequisites, or exclusions, leaving the agent without direction on appropriate usage scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_vlanC
Update VLAN description
| Name | Required | Description | Default |
|---|---|---|---|
| description | Yes | New description | |
| tag | Yes | VLAN tag |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It states 'update' which implies mutation, but doesn't disclose behavioral traits like required permissions, whether changes are reversible, error handling, or side effects. This leaves significant gaps for a mutation tool with no annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It's appropriately sized and front-loaded, making it easy to parse quickly without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given this is a mutation tool with no annotations and no output schema, the description is incomplete. It doesn't explain what the tool returns, error conditions, or behavioral context, leaving the agent with insufficient information for reliable invocation beyond basic parameter passing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with both parameters ('tag' and 'description') documented in the schema. The description adds no additional meaning beyond the schema's parameter descriptions, so it meets the baseline of 3 where the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Update VLAN description' states the action (update) and resource (VLAN description), but is vague about what specifically is being updated. It doesn't distinguish from sibling tools like 'update_firewall_rule' or 'create_vlan', and the title is null, leaving the purpose somewhat unclear beyond basic verb+resource.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., VLAN must exist), exclusions, or comparisons to siblings like 'create_vlan' or 'delete_vlan'. The description offers no context for usage decisions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
64 tool updates
v1.0.0- First observed
apply_blocklist_category - First observed
block_domain - First observed
block_multiple_domains - First observed
configure - First observed
create_backup - First observed
create_firewall_preset - First observed
create_firewall_rule - First observed
create_vlan - First observed
delete_firewall_rule - First observed
delete_vlan - First observed
find_arp_by_hostname - First observed
find_arp_by_interface - First observed
find_arp_by_ip - First observed
find_arp_by_mac - First observed
find_device_by_mac - First observed
find_device_by_name - First observed
find_devices_on_vlan - First observed
find_firewall_rules - First observed
get_arp_stats - First observed
get_devices_by_interface - First observed
get_firewall_rule - First observed
get_guest_devices - First observed
get_interfaces - First observed
get_vlan - First observed
haproxy_acl_create - First observed
haproxy_action_create - First observed
haproxy_backend_create - First observed
haproxy_backend_delete - First observed
haproxy_backend_health - First observed
haproxy_backend_list - First observed
haproxy_certificate_create - First observed
haproxy_certificate_list - First observed
haproxy_frontend_create - First observed
haproxy_frontend_delete - First observed
haproxy_frontend_list - First observed
haproxy_service_control - First observed
haproxy_stats - First observed
iac_apply_deployment - First observed
iac_destroy_deployment - First observed
iac_list_resource_types - First observed
iac_plan_deployment - First observed
list_arp_entries - First observed
list_backups - First observed
list_dhcp_leases - First observed
list_dns_blocklist - First observed
list_firewall_rules - First observed
list_vlans - First observed
macro_analyze - First observed
macro_delete - First observed
macro_export - First observed
macro_generate_tool - First observed
macro_import - First observed
macro_list - First observed
macro_play - First observed
macro_start_recording - First observed
macro_stop_recording - First observed
restore_backup - First observed
search_dns_blocklist - First observed
test_connection - First observed
toggle_blocklist_entry - First observed
toggle_firewall_rule - First observed
unblock_domain - First observed
update_firewall_rule - First observed
update_vlan
TDQS
The tool set has clear groupings by domain (e.g., ARP, firewall, HAProxy, macros), which helps disambiguation within groups, but there is significant overlap across some tools. For example, multiple ARP-related tools (find_arp_by_hostname, find_arp_by_interface, find_arp_by_ip, find_arp_by_mac) have very similar purposes and could be confused, and the macro tools (macro_play, macro_start_recording, macro_stop_recording) have overlapping functionalities. However, descriptions provide enough detail to differentiate them in most cases.
Naming is mostly consistent with a verb_noun pattern (e.g., create_firewall_rule, delete_vlan, list_arp_entries), and snake_case is used throughout. There are minor deviations, such as 'configure' and 'test_connection' which omit the noun, and 'iac_apply_deployment' uses a prefix, but overall the pattern is predictable and readable across the set.
With 64 tools, the count is excessive for a single server, making it feel heavy and potentially overwhelming. While OPNsense is a broad platform, the tools cover multiple distinct domains (e.g., firewall, VLAN, HAProxy, macros, ARP), suggesting they might be better split into separate, more focused servers. This large number increases complexity and reduces coherence.
The tool surface is quite comprehensive for the OPNsense domain, covering core areas like firewall rules (create, get, list, update, delete, toggle), VLAN management, DNS blocklisting, HAProxy configuration, and macro operations. Minor gaps exist, such as no direct tools for DHCP configuration beyond listing leases, but overall, it provides good CRUD/lifecycle coverage and few dead ends for agents.
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Connectors
Hosted MCP server connecting claude.ai, ChatGPT and other AI apps to your own computer
MCP server unifying ERPs, CRMs, APIs and knowledge base for Claude, ChatGPT and Gemini.
- mcpOAuthcom.gibsonai
GibsonAI MCP server: manage your databases with natural language
The Ramp MCP server enables users to securely connect Ramp with AI assistants like ChatGPT and Claude to query financial data and take actions using natural language. It transforms Ramp's developer API into a SQL interface that LLMs can query, allowing admins to analyze spend trends, identify cost savings, and run complex SQL analyses on comprehensive datasets (transactions, purchase orders, vendors, users), while all users can manage cards, view transactions, request reimbursements, and get expense policy answers.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceA Model Context Protocol (MCP) server implementation for managing OPNsense firewalls. This server allows Claude and other MCP-compatible clients to interact with all features exposed by the OPNsense API.1AGPL 3.0
- AlicenseBqualityAmaintenanceA production-grade server that enables natural language interaction with pfSense firewalls through Claude Desktop and other GenAI applications, supporting multiple access levels and functional categories.10094MIT
- -licenseNot gradedqualityNot gradedmaintenanceA production-ready server that connects Claude Desktop to Firewalla network management capabilities, allowing users to monitor devices, analyze network traffic, manage security alerts, and configure firewall rules through natural language.-
- AlicenseNot gradedqualityDmaintenanceEnables natural language management of OPNsense firewalls through AI clients, providing tools for firewall rules, system health, VPN monitoring, and more.AGPL 3.0
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/vespo92/OPNSenseMCP'
If you have feedback or need assistance with the MCP directory API, please join our Discord server