Server Quality Checklist

Tool Scores

• get_recent_commitsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the return format (commit hash, author, date, message) and hints at a use case, but lacks details on permissions, rate limits, error handling, or pagination. It adds some value but does not fully compensate for the absence of annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by a use case hint and return details in just two sentences. Every sentence adds value without redundancy, making it efficient and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description provides basic purpose and return fields but lacks details on behavioral aspects like error handling or performance. It is adequate for a read-only tool but could be more complete to fully guide an agent in complex scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, so the schema already documents all parameters thoroughly. The description does not add any parameter-specific semantics beyond what the schema provides, such as explaining the format of 'since' or the implications of 'show_diff'. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('retrieves') and resource ('recent git commits for the workspace'), and distinguishes this from siblings by specifying it's about git commits rather than debugging, testing, or code search. It also mentions the specific return fields (commit hash, author, date, message), making the purpose highly specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage guidance by stating it's 'useful for correlating code changes with failures,' which suggests a context for when to use it. However, it does not explicitly state when to use this tool versus alternatives like 'search_codebase' or provide any exclusions or prerequisites, leaving some ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_codebaseA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds valuable context beyond basic functionality: it describes the implementation ('Uses ripgrep when available, falls back to a recursive file walk'), which helps set performance expectations. However, it doesn't cover aspects like rate limits, error handling, or authentication needs, leaving some behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the first sentence stating the core functionality and subsequent sentences adding implementation details and return values. Every sentence earns its place by providing essential information without redundancy, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 parameters, no output schema, no annotations), the description is mostly complete. It covers purpose, behavior, and return values, but lacks details on error cases or output format specifics. Since there's no output schema, more clarity on the return structure would be beneficial, though the current description is adequate for basic use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, providing clear documentation for all parameters. The description adds no additional parameter semantics beyond what the schema already covers, such as explaining the 'query' pattern format or 'path' scoping in more detail. This meets the baseline score of 3 since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('search across workspace files') and resources ('codebase'), distinguishing it from sibling tools like get_recent_commits or run_tests. It specifies what type of search it performs ('fast text search') and what it returns ('matching file paths, line numbers, and line content'), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for text searching in code files but provides no explicit guidance on when to use this tool versus alternatives. There are no sibling tools with similar search functionality listed, so no differentiation is needed, but it lacks context about prerequisites or scenarios where this tool is preferred over other methods.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_testsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It successfully describes key behavioral traits: the auto-detection capability, the safety gate mechanism requiring confirmation, and the fact that it executes tests. It doesn't mention rate limits, authentication needs, or what happens on failure, but provides substantial operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with just two sentences that each earn their place. The first sentence establishes the core purpose, and the second adds crucial safety information. No wasted words, and the most important information (the safety requirement) is appropriately emphasized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters, no annotations, and no output schema, the description provides adequate but incomplete context. It covers the main purpose and safety mechanism well, but doesn't explain what the tool returns, how errors are handled, or provide guidance on when to use command overrides versus auto-detection. Given the complexity, more completeness would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the baseline is 3. The description adds some value by emphasizing the critical safety requirement for the 'confirm' parameter ('MUST be set to true to actually execute'), but doesn't provide additional meaning beyond what's already documented in the schema descriptions for the other parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Executes') and resource ('workspace's test suite'), and distinguishes from siblings by specifying it's about running tests rather than debugging, connecting, or searching. It goes beyond just restating the name/title by adding context about auto-detection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool (to run tests in the workspace) and includes an important safety requirement (confirm flag must be true). However, it doesn't explicitly state when NOT to use it or mention alternatives among the sibling tools, which prevents a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• evaluate_expressionA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well. It discloses that the tool requires an active DAP connection and works best when debuggee is stopped. It explains the behavioral difference between 'hover' (safe, no side effects) and 'watch' (read-only but different formatting). However, it doesn't mention potential limitations like expression complexity constraints or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with zero waste. First sentence states purpose, second provides prerequisites and optimal conditions, third explains the key parameter distinction. Every sentence earns its place and information is front-loaded appropriately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a debug expression evaluation tool with no annotations and no output schema, the description does well. It covers prerequisites, optimal conditions, and key parameter semantics. However, without an output schema, it doesn't describe what the evaluation returns (values, errors, formats), which would be helpful for completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds some value by explaining that 'context' restricts evaluation type and distinguishing between 'hover' and 'watch', but doesn't provide additional semantic context beyond what's already in the schema descriptions. The schema already documents all parameters thoroughly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'evaluates' and resource 'expression' with context 'in the context of the debuggee's current state'. It distinguishes from siblings like 'step_debugger' or 'inspect_runtime_state' by focusing specifically on expression evaluation rather than debugging control or state inspection. However, it doesn't explicitly contrast with all possible alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance: 'Requires an active DAP connection and typically works best when the debuggee is stopped.' It also distinguishes between 'hover' (default, safe) and 'watch' contexts, giving clear operational context. The prerequisites are clearly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• step_debuggerA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behavioral traits: it 'mutates the state of the debuggee' (indicating a write operation), specifies a prerequisite ('established debugging connection which is currently paused'), and mentions a timeout behavior ('wait up to 3 seconds for it to pause again'). This covers critical aspects like mutation, preconditions, and timing, though it doesn't address potential errors or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the first sentence stating the core purpose and the second adding crucial behavioral context. Every sentence earns its place by providing essential information without redundancy, making it efficient and easy to parse for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (mutating debuggee state with timing constraints), no annotations, and no output schema, the description does a good job of covering key aspects like purpose, prerequisites, and behavior. However, it lacks details on return values or error conditions, which would be helpful for a mutation tool with no structured output documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, providing complete documentation for both parameters. The description adds no additional parameter semantics beyond what's in the schema (e.g., it doesn't explain 'thread_id' usage further or provide examples). Given the high schema coverage, the baseline score of 3 is appropriate as the description doesn't compensate but also doesn't detract.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('controls execution flow', 'resume, step over, step into, or step out') and identifies the resource ('debuggee'). It distinguishes itself from sibling tools like 'connect_dap', 'disconnect_dap', and 'evaluate_expression' by focusing on execution control rather than connection management or expression evaluation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit context for when to use this tool ('Requires an established debugging connection which is currently paused'), which clearly differentiates it from tools like 'connect_dap' or 'initialize_session'. However, it doesn't specify when NOT to use it or mention alternatives for similar actions, such as how 'run_tests' might relate to execution flow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• connect_dapA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It effectively describes the tool's behavior as establishing a TCP connection and mentions prerequisites (debug adapter must already be listening). However, it doesn't specify what happens on failure, whether the connection persists, or any authentication requirements, leaving some behavioral aspects unclear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with three sentences that each serve distinct purposes: stating the core function, specifying prerequisite relationships with sibling tools, and providing concrete usage examples. There's no wasted language, and the most critical information (what the tool does and when to use it) appears first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a connection-establishing tool with 5 parameters (1 required) and no annotations or output schema, the description provides good context about purpose, prerequisites, and sibling relationships. However, it doesn't describe what happens after connection (e.g., connection state management, error handling, or what the tool returns), which would be helpful given the lack of output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents all 5 parameters. The description doesn't add any parameter-specific information beyond what's in the schema. It mentions 'specified host and port' generically but provides no additional syntax, format, or usage details about parameters. This meets the baseline expectation when schema coverage is complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Establishes a TCP connection') and target resource ('to a running Debug Adapter Protocol server'). It explicitly distinguishes this tool from sibling tools like 'inspect_runtime_state' and 'evaluate_expression' by stating it must be called before those tools, establishing a clear hierarchy and purpose differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool ('before `inspect_runtime_state` or `evaluate_expression`') and prerequisites ('The debug adapter must already be listening on the specified host and port'). It gives concrete examples of when the debug adapter would be listening (Node.js app with `--inspect-brk`, VS Code debug adapter in server mode), offering clear context for appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• initialize_sessionA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's behavior ('auto-detects' multiple workspace aspects) and its role in establishing context, though it lacks details on potential errors, performance, or output format, which would be beneficial given the absence of annotations and output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with two sentences that efficiently convey the tool's purpose and usage without any wasted words, making it easy for an AI agent to parse and apply.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (context establishment with auto-detection), no annotations, and no output schema, the description is largely complete for its purpose. It clearly states what the tool does and when to use it, though it could benefit from mentioning output expectations or error handling to fully compensate for the lack of structured data.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single parameter ('workspace_path'). The description does not add any parameter-specific information beyond what the schema provides, resulting in a baseline score of 3 as the schema handles the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool's purpose with specific verbs ('auto-detects') and resources ('technology stack, package manager, test runner, log paths, and entry points'), clearly distinguishing it from sibling tools like 'connect_dap' or 'run_tests' by focusing on initial context establishment rather than debugging actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool ('Call this first to establish context before debugging'), including a clear alternative context (use before debugging) and an implicit exclusion (not for ongoing debugging tasks like 'step_debugger').

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inspect_runtime_stateA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does well by disclosing prerequisites (active DAP connection), optimal usage context (debuggee stopped at breakpoint), and scope of data retrieved. It doesn't mention rate limits, error handling, or performance impacts, but covers key behavioral aspects for a debugging tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place: first states purpose and data scope, second gives prerequisites, third provides usage context. Front-loaded with core functionality, no wasted words, and structured logically from general to specific.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a debugging tool with no annotations and no output schema, the description is quite complete—it covers purpose, prerequisites, optimal usage, and data scope. It doesn't describe the return format or error cases, but given the context and sibling tools, it provides sufficient guidance for an agent to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds no additional parameter semantics beyond what's in the schema, but the baseline is 3 since the schema does the heavy lifting. No compensation is needed given high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Retrieves') and resource ('live debugging snapshot from a connected DAP session'), specifying what data is obtained (threads, call stack, scopes, local variables with values). It distinguishes from siblings like 'connect_dap' (prerequisite) and 'step_debugger' (action vs. inspection).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use ('Most useful when the debuggee is stopped at a breakpoint') and prerequisites ('Requires an active DAP connection (call `connect_dap` first)'). It also implies alternatives by naming a sibling tool for the prerequisite, providing clear context for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• disconnect_dapA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behavioral traits: the debuggee continues running (non-destructive), the tool is idempotent (safe for repeated calls), and it handles disconnection cleanly. This provides clear context beyond basic functionality.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded: the first sentence states the core purpose, followed by clarifying sentences that add critical behavioral context. Every sentence earns its place with no wasted words, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (simple disconnection with no parameters), no annotations, and no output schema, the description is complete. It covers purpose, usage guidelines, behavioral traits (non-termination, idempotency), and confirms no parameters needed, providing all necessary context for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so the baseline is 4. The description doesn't need to explain parameters, but it implicitly confirms no parameters are needed by describing the action without mentioning any inputs, which aligns perfectly with the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('cleanly disconnects') and resource ('current DAP session'), distinguishing it from sibling tools like 'connect_dap' or 'terminate' operations. It precisely defines what the tool does without being tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool ('cleanly disconnects from the current DAP session') and when not to use it ('The debuggee continues running — this does NOT terminate it'). It also provides an alternative context by mentioning idempotency, which helps differentiate from tools that might terminate sessions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.