Server Quality Checklist

Tool Scores

• index_projectA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states 'forces a graph rebuild' and returns a status dict, implying a mutation. However, it does not disclose whether the operation is idempotent, requires specific permissions, or has any side effects on the project files. The description is adequate but lacks depth on behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, uses a clear structure with an opening statement, usage guidance, args, returns, and an example. Every sentence adds value and there is no redundancy. It is well front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one required parameter, no output schema, no nested objects), the description covers the essential aspects: purpose, when to use, parameter explanation, return type, and an example. It lacks error scenarios or format validation, but overall it is complete enough for an AI agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It explains project_root as 'Absolute path to the project root directory.' This adds basic meaning but does not specify constraints like path must exist, format, or that the project must already be recognized. The description is minimal and barely adds value beyond the parameter name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool builds or refreshes the dependency graph for a project. It specifies the verb 'build or refresh' and resource 'dependency graph for a project'. It also distinguishes itself from the sibling tool 'assess_command' by noting that this tool forces a rebuild, while assess_command auto-builds on first use.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides usage guidance: 'Normally not required — assess_command auto-builds the graph on first use — but useful to refresh after a large code change.' This tells the agent when to use this tool and when not to, directly addressing the alternative sibling tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• restore_snapshotA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It states 'Overwrites whatever currently exists at each snapshotted path,' disclosing destructive behavior. It also mentions return format. However, it lacks details on permissions, atomicity, or error handling for missing paths.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: front-loaded purpose, followed by args, returns, and an example. Every sentence adds value with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 required params, no output schema), the description covers purpose, parameter semantics, return format, usage context, and provides an example. It is complete for an agent to understand and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description adds meaning: snapshot_id is described as 'The snapshot id to restore' and project_root as 'The project root the snapshot was taken under.' The example shows a timestamp format for snapshot_id, going beyond schema titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Undo a risky command by restoring a snapshot's files in place.' The verb 'restoring' and resource 'snapshot' are specific. Siblings like list_snapshots and assess_command are distinct in function, so no confusion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises to 'Use list_snapshots to find the id,' guiding the agent to a prerequisite step. It implies usage after a risky command. However, it does not explicitly exclude alternative scenarios or mention when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_snapshotsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description fully carries the transparency burden. It discloses that snapshots are automatic, stored under a specific directory, and sorted newest first. It also details the return format. No side effects are needed for a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with no wasted words. It uses a docstring format with clear sections (Args, Returns, Example) and front-loads the purpose. Every sentence provides value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple listing tool with one parameter and no output schema, the description covers purpose, trigger, location, ordering, and return format, plus an example. It is practically complete, though it could mention that restore_snapshot is the complementary sibling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'project_root' is described as 'The project root the snapshots were taken under,' which adds meaningful context beyond the schema's generic 'Project Root' title. With schema description coverage at 0%, the description compensates well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'List undo snapshots captured before risky commands, newest first.' This clearly defines the verb (list), resource (undo snapshots), and ordering. It distinguishes itself from siblings like restore_snapshot by focusing on listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that snapshots are taken automatically before medium-or-higher risk commands, giving clear context for when to use this tool. It does not explicitly state when not to use it, but the purpose is well-defined.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• assess_commandA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses key behaviors: splits chains on separators, resolves paths, scores independently, auto-builds graph when project_root provided. No annotation provided, so description carries full burden; could mention if there are side effects beyond graph building.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise yet comprehensive: first sentence states purpose, then details on chain processing, project_root behavior, Args/Returns/Example. Every sentence serves a clear purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description explains return structure (worst-step score, severity, recommendation, per-step chain). Example and parameter details make it complete for agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but description adds rich meaning: command as raw string, cwd for relative path resolution, project_root triggers auto-graph building. Far exceeds schema's types and names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool assesses the blast radius of a shell command, splitting chained commands and scoring each segment. It distinguishes from sibling tools like index_project by focusing on analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides context on when to use (assessing blast radius) and explicitly mentions one alternative (use index_project to force rebuild). Lacks explicit 'when not to use', but example and parameter details guide usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.