Server Quality Checklist

Tool Scores

• vpn_connectC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the action but gives no detail on side effects, such as whether connecting interrupts an existing VPN session, whether it blocks until the connection is established, or what happens on failure. It does not explicitly contradict annotations, but it is far from sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally brief (two sentences, six words) and wastes no words. It front-loads the core purpose and mentions the only parameter. However, it is slightly too terse to serve as a complete guide, though the conciseness itself is appropriate.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that this is a network-changing tool with no annotations, no output schema, and only one undocumented parameter, the description is too sparse to be considered complete. It lacks information about the connection behavior, prerequisites, fallback options, and how it relates to sibling tools like vpn_require_active. A description of this length leaves critical gaps for an agent to invoke the tool safely and correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one parameter 'country' with no description; coverage is 0%. The description says 'Optional argument: country,' which adds the fact that it is optional (though schema already implies this via 'required: 0') and identifies its purpose as a country selector. However, it does not provide valid values, expected format (e.g., two-letter code vs. full name), or example inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Connect') and the resource ('NordVPN'), which is specific and unambiguous. It naturally distinguishes from siblings like vpn_status, vpn_disconnect, and vpn_public_ip by indicating a connection-establishing operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The only guidance is 'Optional argument: country,' which indicates a parameter but provides no context on when to use this tool versus alternatives. It does not mention that vpn_status is for checking connection, vpn_disconnect for terminating, or any prerequisites like having config files or authenticated sessions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vpn_statusA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotation readOnlyHint=true already signals this is a safe read operation, so the description adds no additional behavioral context. It does not disclose what the status output looks like, whether it can fail when disconnected, or any other nuances, providing minimal value beyond the structured annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short sentence that delivers exactly the needed information. Every word earns its place with no redundancy or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, the description is adequate, but it lacks any explanation of the return value format (e.g., boolean, string). Since there is no output schema, the description should clarify what 'connection status' means to fully prepare the agent, but the one-line description is still a minimal viable explanation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema is empty, so there is nothing to document. Baseline 4 applies because no parameter explanation is needed, and the description does not introduce any ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a clear verb ('Return') and a specific resource ('NordVPN connection status'), which precisely states the tool's function. It distinguishes from siblings like vpn_connect/vpn_disconnect (actions) and vpn_public_ip (a different data point), leaving no ambiguity about what is checked.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives. The description does not mention sibling tools or provide context such as 'use to verify connectivity before other operations,' leaving the agent without explicit direction on when this tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vpn_disconnectA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds no behavioral details beyond the annotations. While idempotentHint and destructiveHint are declared, the description doesn't explain what disconnecting entails (e.g., terminates the active session, safe when already disconnected), so it fails to enrich the annotation-provided safety profile.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The one-sentence description is extremely concise and front-loaded, with every word contributing to clarity. There is no redundancy or filler, making it an efficient and effective statement of purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with no output schema, the description is minimally sufficient. However, it doesn't state what happens if no VPN connection is active, though the idempotentHint annotation indirectly covers this, so the completeness is adequate but not exhaustive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema is fully covered by default. Per the rubric, a baseline of 4 is appropriate when no parameters exist, and the description adds no unnecessary parameter-related text.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Disconnect NordVPN' clearly specifies the action (disconnect) and the target resource (NordVPN), making it unambiguous and distinct from sibling tools like vpn_connect and vpn_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It lacks context about prerequisites, typical scenarios, or exclusions, leaving the agent to infer usage from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vpn_public_ipA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotation readOnlyHint: true already indicates a safe read operation. The description's 'Return' aligns with that and adds no additional behavioral context (e.g., no mention of VPN-specific behavior, possible latency, or errors). It does not contradict the annotation, but it also does not provide extra transparency beyond what the annotation and the basic description show.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that directly states the purpose without any filler or redundant information. It is front-loaded and easy to parse, making it an model of efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters, no output schema, and a simple read-only operation, the description provides complete context: it clearly states what the tool returns (the current public IP address). The tool name and sibling names provide the VPN context, so the description is sufficiently complete for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description is not required to explain parameter semantics. The description implicitly conveys the return value (the IP address) by stating 'Return current public IP address', which is useful given the absence of an output schema. Following the baseline of 4 for zero-parameter tools, this is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action with a specific verb ('Return') and resource ('current public IP address'). It is easily distinguishable from sibling tools like vpn_status, vpn_connect, vpn_disconnect, and vpn_require_active, which focus on connection state and control.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: if you need the current public IP address, you use this tool. However, it does not explicitly state when to use it versus alternatives or provide any contextual exclusions. The simple nature of the tool makes the implied usage fairly clear, but there is no explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vpn_require_activeA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint annotation already indicates a safe, side-effect-free operation. The description adds behavioral context by disclosing that the tool fails when the VPN is not connected, which is beyond the annotation. It does not detail the exact failure mode but provides essential gating behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that immediately conveys the tool's function without any filler. It is perfectly front-loaded and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter assertion tool with a read-only annotation, the description is contextually complete. It tells the agent exactly when it fails (when VPN is not connected) and implicitly when it passes. No output schema or return value details are necessary for this simple guard behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters, the baseline score of 4 applies. There is no parameter information to add, and the description is not required to compensate for any missing schema details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: it fails unless NordVPN is connected. The verb 'fail' and resource 'NordVPN' make the purpose unmistakable, and it is clearly distinct from siblings like vpn_status (which reports status) and vpn_connect (which establishes a connection).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage is implied as a guard for operations requiring an active VPN, but the description does not explicitly say when to use it versus alternatives like vpn_status, nor does it mention exclusions or prerequisites. There is no alternative guidance, so it only meets the 'implied usage' level.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.