Server Quality Checklist

Tool Scores

• device_actionC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states 'Perform actions' which implies mutation operations, but doesn't disclose any behavioral traits like whether actions are destructive, require specific permissions, have side effects, or what happens after execution. For a tool with potentially destructive actions (like 'delete'), this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just 5 words: 'Perform actions on a specific device'. It's front-loaded with the core purpose and has zero wasted words. Every word earns its place in conveying the essential function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a mutation tool with potentially destructive actions (delete, deauthorize) and no annotations or output schema, the description is incomplete. It doesn't explain what the actions do, their consequences, or what to expect as results. For a tool that could permanently affect devices, more context is needed about behavior and outcomes.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters (deviceId and action with enum values). The description adds no additional meaning beyond what the schema provides - it doesn't explain what each action does, consequences of choices, or parameter relationships. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the purpose as 'Perform actions on a specific device', which is clear but vague. It specifies the verb 'perform actions' and resource 'device', but doesn't distinguish what types of actions or differentiate from sibling tools like 'list_devices' or 'manage_device_tags'. The purpose is understandable but lacks specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With many sibling tools available (like manage_device_tags, manage_keys, etc.), there's no indication of when device_action is appropriate versus other device-related tools. No context, exclusions, or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manage_aclC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure but offers minimal insight. 'Manage' implies mutation capabilities, but it doesn't specify whether this requires admin permissions, what side effects occur (e.g., network disruption), or how operations like 'validate' behave. For a tool with complex ACL operations, this is inadequate transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's appropriately sized for a tool name that already includes 'manage_acl', though this conciseness comes at the cost of detail. Every word earns its place by identifying the resource being managed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 2 parameters (one being a complex nested object), no annotations, and no output schema, the description is insufficiently complete. It doesn't address what the tool returns, error conditions, or the impact of different operations. The agent lacks context about how this tool fits into the broader Tailscale management ecosystem.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description adds no additional meaning about parameters beyond what's in the schema. It doesn't explain the relationship between 'operation' and 'aclConfig' or provide usage examples. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool manages Tailscale Access Control Lists (ACLs), which provides a basic purpose but lacks specificity about what 'manage' entails. It doesn't distinguish this from sibling tools like manage_policy_file or manage_network_lock that also handle network policies. The verb 'manage' is vague without clarifying which operations are supported.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus alternatives. The description doesn't mention prerequisites, appropriate contexts, or exclusions. Given sibling tools like manage_policy_file that might overlap with ACL functionality, the absence of differentiation leaves the agent without usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manage_dnsC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. 'Manage' implies both read and write operations, but it doesn't specify permissions needed, side effects, rate limits, or response format. For a tool with multiple mutation operations, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's appropriately sized for a tool name that suggests its domain, though it could be more informative given the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 parameters, multiple mutation operations, no annotations, and no output schema, the description is inadequate. It doesn't explain the tool's scope, behavioral traits, or return values, leaving significant gaps for the agent to navigate this complex configuration tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all parameters. The description adds no additional parameter semantics beyond what's in the schema descriptions (e.g., which parameters correspond to which operations). This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Manage Tailscale DNS configuration' states the general purpose (managing DNS configuration) but is vague about what specific operations are available. It doesn't distinguish this tool from potential DNS-related siblings (though none are listed), and the verb 'manage' is broad rather than specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, context, or exclusions, leaving the agent to infer usage from the operation parameter alone without any directional advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manage_exit_nodesC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. 'Manage' implies both read and write operations, but it doesn't specify permissions required, side effects, or what happens during operations like 'set' or 'clear'. It mentions 'exit nodes and routing' but gives no insight into rate limits, authentication needs, or system impact. The description is too generic to adequately inform behavioral expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—just five words—with zero wasted language. It's front-loaded with the core concept ('Manage Tailscale exit nodes and routing') and contains no unnecessary elaboration. This efficiency makes it easy to parse, though it may be too brief for complete understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multiple operations including destructive actions like 'set' and 'clear'), lack of annotations, and no output schema, the description is insufficiently complete. It doesn't explain what the tool returns, error conditions, or operational consequences. For a tool with potentially significant network impact, more context about behavior and outcomes is needed beyond the bare statement provided.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all three parameters (deviceId, operation, routes). The description adds no additional parameter semantics beyond what's in the schema—it doesn't explain parameter relationships, provide examples beyond the schema's route example, or clarify when deviceId is needed. With high schema coverage, the baseline score of 3 is appropriate as the description doesn't compensate but doesn't detract either.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Manage Tailscale exit nodes and routing' states the general domain (exit nodes and routing) but lacks specificity about what 'manage' entails. It doesn't distinguish this tool from sibling tools like 'manage_routes' or 'manage_acl', leaving the exact verb and resource scope vague. However, it does identify the correct technology context (Tailscale).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With sibling tools like 'manage_routes', 'device_action', and 'list_devices', there's no indication of how this tool differs or when it should be preferred. The description offers only a generic statement without context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manage_file_sharingC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. 'Manage' implies mutation capabilities, but it doesn't specify permissions required, side effects, or response format. The description fails to explain that operations include getting status, enabling, and disabling (which the schema reveals), leaving the agent unaware of the tool's full behavioral range.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's appropriately sized for a tool with only two parameters and clear scope. No unnecessary details or repetition are present, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a mutation-capable tool. It doesn't cover behavioral aspects like what 'enable' or 'disable' do, error conditions, or return values. For a tool that can change system state, more context is needed to use it safely and effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters (deviceId and operation with enum values). The description adds no additional meaning about parameters beyond what's in the schema, such as explaining when deviceId is required or the implications of each operation. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Manage Tailscale file sharing settings' states the general purpose (managing file sharing settings) but lacks specificity about what 'manage' entails. It doesn't distinguish this tool from sibling tools like 'manage_acl' or 'manage_policy_file' that also manage different Tailscale settings. The verb 'manage' is vague compared to more precise alternatives like 'configure' or 'control'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There's no mention of prerequisites, context for file sharing operations, or comparison to sibling tools. The agent must infer usage from the tool name and parameters alone, which is insufficient for clear decision-making.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manage_keysC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. 'Manage' implies mutation capabilities, but the description doesn't specify what operations are available (list, create, delete), what permissions are required, whether operations are destructive, or what the response format looks like. For a tool with mutation capabilities and no annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at just 4 words with no wasted language. It's front-loaded with the core purpose. While it's arguably too brief for a tool with mutation capabilities, it earns full marks for conciseness as every word contributes to the core message.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with mutation capabilities (create, delete operations), no annotations, no output schema, and complex nested parameters, the description is inadequate. It doesn't explain the available operations, their effects, return values, or error conditions. The agent would need to rely entirely on the input schema to understand how to use this tool, which is insufficient for safe operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 3 parameters (operation, keyConfig, keyId) with their purposes. The description adds no additional parameter semantics beyond what's in the schema. According to the rules, when schema coverage is high (>80%), the baseline is 3 even with no param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Manage Tailscale authentication keys' states the resource (authentication keys) but uses the vague verb 'manage' without specifying what operations are available. It doesn't differentiate from sibling tools like manage_acl or manage_device_tags, which also 'manage' different resources. The purpose is understandable but lacks specificity about the CRUD operations involved.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, dependencies, or scenarios where this tool is appropriate. Given the sibling tools include various management functions, there's no indication of how this tool fits into the broader context of Tailscale administration.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• connect_networkC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. 'Connect to the Tailscale network' implies a state-changing operation but doesn't specify whether this requires admin privileges, what happens if already connected, whether it's idempotent, what side effects occur, or what the expected outcome looks like. For a network connection tool with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's appropriately sized for a tool with this level of complexity and gets straight to the point without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a network connection tool with 6 parameters, no annotations, and no output schema, the description is incomplete. It doesn't explain what 'connect' means operationally, what happens after connection, what permissions are required, or what the tool returns. The agent must rely entirely on the input schema for parameter details and guess at behavioral aspects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 6 parameters thoroughly. The description adds no parameter-specific information beyond what's in the schema. According to scoring rules, when schema coverage is high (>80%), the baseline is 3 even with no param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Connect') and the target resource ('Tailscale network'), providing a specific verb+resource combination. However, it doesn't differentiate this tool from its sibling 'disconnect_network' beyond the obvious opposite action, nor does it specify what 'connect' entails operationally beyond the high-level concept.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There's no mention of prerequisites (e.g., needing authentication), when this operation is appropriate versus using other network management tools, or what state the network should be in before invocation. The agent must infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_network_statusC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. While 'Get' implies a read operation, the description doesn't disclose important behavioral traits like whether this requires authentication, has rate limits, what the output looks like, or if it's a real-time status check versus cached data. The mention of 'Tailscale CLI' hints at the underlying implementation but doesn't clarify operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any unnecessary words. It's appropriately sized for a simple tool and front-loads the essential information immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a network status tool with no annotations and no output schema, the description is insufficient. It doesn't explain what 'network status' includes (connectivity, peers, routes, etc.), the format or structure of returned data, or any prerequisites or limitations. Given the complexity of network diagnostics and the lack of structured metadata, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents the single 'format' parameter with its enum values and description. The description adds no additional parameter information beyond what's in the schema, which meets the baseline expectation when schema coverage is complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('current network status from Tailscale CLI'), making the purpose immediately understandable. It doesn't explicitly distinguish from siblings like 'get_tailnet_info' or 'list_devices', but the specificity of 'network status' provides reasonable differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With siblings like 'get_tailnet_info', 'list_devices', and 'ping_peer' available, there's no indication of what distinguishes network status from other network-related queries or when this specific tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_tailnet_infoC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool retrieves information (implying read-only), but doesn't address critical aspects like authentication requirements, rate limits, error conditions, or what 'detailed information' entails beyond the parameter. This leaves significant gaps for safe and effective use.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any redundant or verbose language. It's appropriately front-loaded and wastes no words, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of network information tools and the lack of annotations or output schema, the description is insufficient. It doesn't explain what 'detailed information' includes, how it differs from sibling tools, or what the return format looks like, leaving the agent with inadequate context for reliable tool selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the parameter 'includeDetails' clearly documented in the schema. The description adds no additional parameter information beyond implying that the tool provides 'detailed' information, which loosely relates to the parameter but doesn't enhance understanding of its semantics or usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('detailed Tailscale network information'), making the purpose immediately understandable. It doesn't explicitly differentiate from sibling tools like 'get_network_status' or 'list_devices', which prevents a perfect score, but the specificity of 'detailed... information' provides some implicit distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_network_status' or 'list_devices'. It lacks any mention of prerequisites, context, or exclusions, leaving the agent to infer usage based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_devicesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action 'List all devices' but doesn't mention any behavioral traits such as pagination, rate limits, authentication requirements, or what the output format looks like (e.g., JSON array, list of objects). This is a significant gap for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any fluff or redundancy. It's appropriately sized and front-loaded, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of listing network devices and the lack of annotations and output schema, the description is incomplete. It doesn't address key contextual aspects like output format, error handling, or how the tool interacts with other device-related operations (e.g., filtering or sorting), leaving gaps for the agent to navigate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the parameter 'includeRoutes' clearly documented as a boolean to include route information. The description doesn't add any meaning beyond the schema, as it doesn't mention parameters at all. With high schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and the resource 'all devices in the Tailscale network', providing a specific purpose. However, it doesn't distinguish this tool from potential siblings like 'device_action' or 'manage_device_tags', which might also involve devices but serve different functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With siblings like 'get_network_status' or 'manage_device_tags', there's no indication of when listing devices is appropriate versus checking status or managing tags, leaving the agent to infer usage from tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manage_device_tagsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'manage' which implies mutation capabilities, but doesn't specify permissions required, whether operations are reversible, rate limits, or what happens with conflicting tags. The 'for organization and ACL targeting' hint provides some context about use cases, but lacks operational details needed for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states the core purpose without unnecessary words. It's appropriately sized for a tool with three parameters and clear schema documentation. Every word earns its place, and the structure is front-loaded with the essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with three parameters, no annotations, and no output schema, the description is inadequate. It doesn't explain what the tool returns, error conditions, or important behavioral aspects like whether 'set_tags' replaces all existing tags or merges with them. The 'for organization and ACL targeting' provides some context, but doesn't compensate for the missing operational details needed for proper tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema. It mentions 'device tags' generally but doesn't explain the relationship between the operation parameter and tags parameter, or provide examples beyond what the schema already shows. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'manage' and resource 'device tags', with the purpose 'for organization and ACL targeting' providing context. It distinguishes from siblings like 'manage_acl' or 'list_devices' by focusing specifically on device tags rather than broader ACL or device operations. However, it doesn't explicitly differentiate from all siblings (e.g., 'device_action' could potentially overlap).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to choose 'manage_device_tags' over 'manage_acl' for ACL-related tasks or 'device_action' for other device operations. There are no prerequisites, exclusions, or contextual usage instructions provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manage_network_lockC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While 'manage' implies mutation capabilities, the description doesn't specify which operations require special permissions, whether changes are reversible, what the tool returns, or any rate limits. The mention of 'enhanced security' hints at authorization needs but lacks concrete details about behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that gets straight to the point. It's appropriately sized for a tool with 3 parameters and no output schema. There's no wasted language, though it could potentially benefit from slightly more detail given the complete lack of annotations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters, 100% schema coverage, but no annotations and no output schema, the description is minimally adequate. It identifies the domain (network lock/key authority) and purpose (enhanced security), but doesn't provide enough context about what the tool returns or the implications of different operations. Given the security-sensitive nature implied by 'key authority', more completeness would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the baseline is 3. The description adds value by clarifying that this manages 'network lock (key authority)', which provides context about what the parameters control beyond what the schema descriptions say. This helps the agent understand that keyId and publicKey parameters relate to key authority management, not just generic operations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool manages Tailscale network lock for enhanced security, which provides a general purpose (verb+resource). However, it's somewhat vague about what 'manage' entails and doesn't clearly differentiate from sibling tools like manage_keys or manage_acl that also handle security-related operations. The description lacks specificity about the key authority aspect mentioned in parentheses.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With multiple 'manage_' sibling tools (manage_acl, manage_keys, manage_device_tags, etc.), there's no indication of when network lock operations are appropriate versus other security or management tools. No context about prerequisites, timing, or exclusions is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manage_policy_fileC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'manage' and 'test' operations but doesn't specify permissions required, whether updates are destructive, rate limits, or what the tool returns. For a tool with multiple operations including updates, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two clear clauses, though it could be more front-loaded by specifying the three operations upfront. Every word earns its place, but the structure could better highlight the tool's multi-operation nature.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multiple operations including updates), lack of annotations, and no output schema, the description is incomplete. It doesn't explain what happens during operations, return values, or error conditions, leaving significant gaps for the agent to understand tool behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value by hinting at policy content format (HuJSON) and access testing, but doesn't provide additional semantics beyond what's in the schema. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as managing policy files and testing ACL access rules, which is specific (verb+resource). However, it doesn't distinguish this from the sibling 'manage_acl' tool, which appears to handle similar ACL-related functionality, preventing a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'manage_acl' or other sibling tools. It lacks context about prerequisites, exclusions, or specific scenarios where this tool is appropriate, leaving the agent with minimal usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manage_routesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While 'enable or disable' implies mutation, it doesn't specify whether this requires admin permissions, if changes are reversible, potential side effects on network connectivity, or error conditions. For a tool that modifies device routing, this leaves critical operational context undocumented.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It's appropriately sized for a tool with clear parameters and gets straight to the point with zero wasted verbiage.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is insufficient. It doesn't explain what happens after enabling/disabling routes (e.g., confirmation message, error responses, network impact) or provide context about typical use cases. Given the complexity of network routing changes, more operational guidance would be expected.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with all parameters well-documented in the schema itself. The description adds no additional parameter context beyond what's already in the schema (device ID, routes as CIDR array, action as enable/disable). This meets the baseline for high schema coverage but doesn't provide extra value like format examples or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('enable or disable') and resource ('routes for a device'), making the purpose immediately understandable. However, it doesn't explicitly differentiate this tool from siblings like 'manage_acl' or 'manage_device_tags', which also manage device configurations, leaving some ambiguity about scope boundaries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With siblings like 'manage_acl' (for access control) and 'device_action' (generic device operations), there's no indication of whether this is for network routing specifically or how it relates to other management tools. No prerequisites or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manage_webhooksC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Manage' implies CRUD operations, but the description doesn't specify what 'manage' entails, what permissions are required, whether operations are destructive, or what the response format looks like. For a multi-operation tool with no annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's appropriately sized and front-loaded with the essential information about what the tool does.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a multi-operation tool (list/create/delete/test) with no annotations and no output schema, the description is incomplete. It doesn't explain the different operations, their effects, or what results to expect. The context signals show this is a complex tool with nested objects and multiple operations that needs more comprehensive documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description adds no parameter-specific information beyond what's in the schema. The baseline of 3 is appropriate when the schema does the heavy lifting for parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('manage') and resource ('Tailscale webhooks for event notifications'), providing a specific purpose. However, it doesn't differentiate this tool from its many siblings on the server, which all appear to be Tailscale management tools with similar naming patterns.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With 17 sibling tools on the server including other 'manage_' tools, there's no indication of when webhook management is appropriate versus other Tailscale management operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ping_peerC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states the basic action without disclosing behavioral traits. It doesn't mention what 'ping' entails (e.g., ICMP echo requests, latency measurement, success/failure criteria), potential side effects, or network requirements, which is insufficient for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's appropriately sized and front-loaded, directly stating the tool's purpose without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (network operation with no annotations and no output schema), the description is incomplete. It lacks details on what the ping operation returns (e.g., latency stats, success indicators) or how it behaves in different network conditions, making it inadequate for the context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting both parameters (target and count). The description adds no additional parameter semantics beyond what the schema provides, so it meets the baseline score of 3 for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Ping a peer device' clearly states the action (ping) and target (peer device), which is specific and unambiguous. However, it doesn't differentiate from sibling tools like 'connect_network' or 'get_network_status', which might also involve network connectivity testing, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, context (e.g., network diagnostics), or exclusions, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• disconnect_networkB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While 'Disconnect from' implies a state-changing operation, it doesn't specify whether this requires authentication, affects other devices, is reversible, or has any side effects. This is inadequate for a tool that likely changes network connectivity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with zero wasted words. It's appropriately sized for a simple tool with no parameters and gets straight to the point without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a network disconnection tool with no annotations and no output schema, the description is insufficient. It doesn't explain what happens after disconnection, whether there are confirmation steps, what the expected outcome is, or how this differs from other network management operations among the many sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema already fully documents the lack of parameters. The description appropriately doesn't discuss parameters, maintaining focus on the tool's purpose. Baseline for 0 parameters is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Disconnect from') and resource ('the Tailscale network'), providing a specific verb+resource combination. However, it doesn't differentiate from sibling tools like 'connect_network' beyond the obvious opposite action, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, consequences, or when it might be appropriate versus other network management tools like 'manage_network_lock' or 'device_action'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_versionB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states what the tool does but doesn't reveal any behavioral traits such as whether it's read-only, requires authentication, has rate limits, or what the output format might be. For a tool with zero annotation coverage, this is inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and uses clear language. Every word earns its place, making it easy to parse and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is minimally adequate. It states the purpose but lacks context about when to use it, behavioral traits, or output details. For a basic read operation, it's passable but could be more helpful with additional guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100%, so there's no need for parameter documentation in the description. The description correctly doesn't mention any parameters, which is appropriate. A baseline of 4 is given since no parameters exist, and the description doesn't introduce confusion.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('Tailscale version information'), making it immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_network_status' or 'get_tailnet_info', which might also retrieve system information. The description is accurate but lacks sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention context, prerequisites, or exclusions, leaving the agent to infer usage based on the tool name alone. This is a significant gap for a server with multiple 'get' tools that could overlap in functionality.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.