Server Quality Checklist

Tool Scores

• snapshot_createC
  2.2/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description 'Create a file manifest snapshot' implies a write operation, but annotations mark readOnlyHint as true, indicating a read operation. This is a direct contradiction, making the description misleading. No additional behavioral traits are disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (one sentence) but lacks critical detail. It is not overly verbose, yet it sacrifices informativeness for brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of parameter descriptions, an existing output schema, and a misleading annotation/description mismatch, the description is incomplete. It does not adequately prepare an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain the purpose or format of either parameter (workspaceId, reason). It only mentions 'open workspace' implicitly, which is insufficient for understanding parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (Create) and resource (file manifest snapshot), along with the context (for an open workspace). It is specific enough to distinguish from general tools, but does not explicitly differentiate from sibling 'checkpoint_create', which could be similar.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like checkpoint_create or other snapshot-related tools. The description lacks any 'when to use' or 'when not to use' information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verification_runC
  2.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate destructive behavior (destructiveHint: true), and the description adds that it runs inside an open workspace. However, no further behavioral details are provided about side effects, permissions, or what 'structured' entails.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short (one sentence), but it is under-specified for a tool with 6 parameters and an output schema. It lacks crucial details, making it insufficiently informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description does not explain the command structure, return values, or usage context. For a tool that runs verification in a workspace, it is incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description should explain parameters, but it does not. None of the 6 parameters (workspaceId, taskId, command, args, workingDirectory, timeoutMs) are described, leaving the agent without necessary context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool runs a verification command inside a workspace, but it does not differentiate from the sibling tool 'shell_run', which also runs commands. The term 'structured' is vague and does not provide enough distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like 'shell_run' or 'file_read'. There are no explicit when-to-use or when-not-to-use instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• directory_listC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotation readOnlyHint=true already declares the tool as read-only. The description adds no extra behavioral context (e.g., error handling, response format, or consequences). It merely restates the purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence and concise, but it adds little value beyond the name and title. It earns its place but could be more informative without being verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has two parameters, one required, and no parameter descriptions, the description is incomplete. The presence of an output schema does not excuse the lack of input parameter details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage and no parameter explanations in the description, the agent receives no guidance on workspaceId or path. The description fails to compensate for the missing schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'List a directory inside an open workspace' clearly specifies the verb (List), resource (directory), and context (open workspace). It distinguishes well from sibling tools like file_read or file_write.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, nor does it mention when not to use it. It only implies the requirement of an open workspace.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• task_startC
  2.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are all false, so the description carries the burden. It discloses the task is 'in-memory', implying non-persistence, but does not mention error handling, permissions, or side effects beyond starting the task.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one short sentence, very concise and front-loaded. However, it could include more detail without losing conciseness, such as parameter purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given three parameters with no schema descriptions and no annotations, the description is incomplete. It lacks explanation of required workspaceId and objective, and optional constraints. The existence of an output schema helps but is not utilized in the description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain any of the three parameters (workspaceId, objective, constraints). The agent must rely solely on parameter names, which is insufficient for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Start an in-memory task under an open workspace', using a specific verb and resource. It distinguishes from sibling tools like task_update and task_status, though it could be more explicit about the scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. With many sibling tools, explicit when/why-not instructions would help, but none are present.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• git_diffC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  While annotations confirm read-only behavior, the description only adds the '--no-color' flag. It does not explain what the diff compares (e.g., working tree vs index, or vs HEAD), nor does it describe the output format beyond no color.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely short (one sentence), which is efficient but omits important details. It is front-loaded with the core action, but lacks completeness. A slightly longer description would be more useful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one required parameter and an output schema, the description is still incomplete. It fails to explain what the output represents (e.g., unstaged changes, staged changes) and how to interpret it. The existence of an output schema somewhat mitigates this, but the description should still provide high-level context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, workspaceId, has no schema description (0% coverage). The description adds the context 'inside an open workspace', implying the workspace must be open, but provides no further details like whether it is a folder or a repository identifier.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Run'), the specific command ('git diff --no-color'), and the context ('inside an open workspace'). It distinguishes from sibling tools like git_status by naming a different git command.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives (e.g., git_status, drift_check). There is no mention of prerequisites, limitations, or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• task_updateC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate non-readonly and non-destructive, but the description adds no behavioral context beyond 'update'. It does not disclose whether updating status triggers side effects, if notes are appended or replaced, or any required permissions. The description does not contradict annotations, but adds minimal value.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, which is concise but overly minimal for a tool with 3 parameters and mutation behavior. It could be slightly longer to add value without being verbose. Every sentence earns its place, but there is room for essential details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool modifies state, has 3 parameters, an output schema (unexplained), and sibling tools for task management, the description is incomplete. It does not explain return values, the effect on existing notes, or differentiate from task_start. Critical gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must compensate, but it only mentions 'status' and 'note' without explaining their meaning or constraints. taskId is not described. The enum for status is captured in schema, but the description adds no additional semantics or usage context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it updates task status or appends a note, using specific verbs and resource. It distinguishes from sibling tools like task_start and task_status by focusing on modification rather than creation or reading.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs alternatives (e.g., task_status for reading). No prerequisites or conditions for updating tasks are mentioned. The description implies usage but does not exclude scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• drift_checkB
  3/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Description contradicts annotations: readOnlyHint=true but description says 'Create a fresh snapshot', which is a write operation. No disclosure of behavioral traits beyond the process.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences that front-load the core action; could be more efficient but no unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With output schema present, return values need not be explained, but parameters are undocumented and behavioral transparency is poor due to contradiction.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% and description does not explain parameters (workspaceId is inferred, but baselineReason and currentReason are left undefined).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool creates a snapshot and compares it with the previous one for the workspace, specifying verb and resource. It distinguishes from siblings like snapshot_create and workspace_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description implies usage scenarios (first call baseline, subsequent calls compare) but does not provide explicit when-to-use or when-not-to-use guidance relative to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• file_readB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so description's addition of 'UTF-8' and open workspace requirement adds some value but does not cover error handling or behavior with offset/limit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, efficiently worded. Not overly concise to the point of omitting critical context, but could add parameter details without verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Does not describe return value (expected file content) or behavior when file not found. Output schema exists but is not shown, leaving gaps in understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, description does not explain any parameter (workspaceId, path, offset, limit), leaving the agent to rely solely on schema definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the action ('Read'), resource ('UTF-8 file'), and constraint ('inside an open workspace'). Distinguishes from sibling tools like file_write or file_edit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives such as search_text or directory_list. Only implies workspace must be open.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_textB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint: true, indicating a safe read operation. The description adds the 'literal' qualifier, implying exact matching not regex, which is useful. However, no other behavioral traits (e.g., case sensitivity, encoding) are disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, efficient sentence with no unnecessary words. Every part of the description adds value, making it highly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description omits critical context: no mention of optional parameters (path, maxResults), no explanation of output, and no performance or scope constraints. For a search tool with multiple parameters, this is insufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does not explain any of the four parameters (workspaceId, pattern, path, maxResults) beyond the tool's generic purpose. The word 'literal' hints at pattern behavior but provides no actual parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb 'search', the resource 'literal text pattern', and the scope 'inside an open workspace'. It is specific and distinguishes from sibling tools, none of which are search tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. Sibling tools like file_read or git_diff could be relevant for different search needs, but no contextual advice is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• file_writeB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide destructiveHint=true, so the description's 'overwrite' adds minimal transparency. It does not explain the overwrite parameter behavior, what happens to existing files, or required permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, efficient, and front-loaded. However, it omits crucial details that would make the tool safer to use, such as the overwrite parameter's role.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 4 parameters, 0% schema coverage, and a destructive annotation, the description fails to provide sufficient context. It does not explain the overwrite behavior or the return output, assuming an output schema exists but not shown.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not elaborate on any parameter. It only implies content is UTF-8. No details on path format, workspaceId usage, or overwrite semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb (create or overwrite), resource (UTF-8 file), and context (inside an open workspace). It distinguishes from siblings like file_edit (which modifies) and file_read (which reads).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage within an open workspace but does not specify when to use this vs. file_edit or other alternatives. No explicit when-not or exclusion advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• workspace_statusB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description is consistent with annotations (readOnlyHint=true) but adds minimal behavioral detail beyond the annotation. Does not disclose error handling or what happens if workspaceId is invalid.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no redundant words. Efficiently captures the dual functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple parameter and available output schema, the description covers the main behaviors adequately. However, it lacks detail on error states or edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the description effectively explains the parameter's behavior: providing workspaceId returns that workspace, omitting it lists open workspaces. This adds significant meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the dual behavior: return a specific workspace by ID or list open workspaces. It is specific with verb+resource, but does not explicitly differentiate from siblings like workspace_open.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus siblings like workspace_open or workspaceguard_info. The context of when to omit workspaceId is implied but not explicitly stated as a use case.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• workspaceguard_infoB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so the description's 'Return ... summary' aligns but adds minimal behavioral insight beyond the annotation. It does not contradict, but adds little value.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with the core purpose. Every word is necessary and there is no unnecessary verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (0 parameters, output schema exists). The description is minimal but sufficient given the output schema handles return structure. However, it could benefit from a brief hint on what the configuration summary includes.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With no parameters (0, 100% schema coverage), the description does not need to add param details. Baseline score of 4 is appropriate as the schema is fully descriptive.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a WorkspaceGuard runtime configuration summary, using a specific verb and resource. It distinguishes from siblings like workspace_status by the specific 'WorkspaceGuard runtime' context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives (e.g., workspace_status, policy_describe) or when not to use it. The description lacks context for appropriate invocation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• checkpoint_createA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds valuable behavioral context beyond annotations: 'metadata only', 'restore is not implemented', and 'snapshot existence is not validated yet'. These disclose significant behavioral traits not captured in annotations (readOnlyHint=false, destructiveHint=false), helping the agent understand side effects and constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that efficiently conveys the core purpose and key limitations. It is front-loaded with the main action. Could be slightly more structured, but overall concise and readable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters (3 required) and an existing output schema, the description adequately covers purpose and behavioral constraints. However, it lacks context on prerequisites (e.g., workspace existence), error conditions, and how this checkpoint interacts with other tools (e.g., if future restore will use it).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description partially explains 'snapshotId' (existing snapshot id) and 'label' (named checkpoint), but omits workspaceId, taskId, and reason entirely. Two out of five parameters receive semantic context, which is insufficient for a schema-poor tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Record a named checkpoint'), the resource ('for an existing snapshot id'), and adds clarifying details that it is metadata-only and lacks restore/validation. This distinguishes it from sibling tools like snapshot_create (which creates snapshots) and task_* (task operations).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. The description lists limitations (no restore, no validation) but does not specify typical usage contexts or when to prefer other tools like snapshot_create or task_*.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shell_runA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare destructiveHint=true and openWorldHint=true. The description adds 'real local execution' confirming side effects, and implicitly notes no shell interpolation. However, it does not detail specific safety concerns or response format beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences that are front-loaded with the core action. No redundant or unnecessary words. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive, open-world tool, the description is minimal. It mentions workspace context but omits execution environment, return behavior, or common pitfalls. However, the presence of annotations and output schema partially compensates.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 60% (timeoutMs lacks description in schema). The tool description does not add new parameter info beyond what is in the schema. The phrase 'structured command array' reinforces use of args, but that is already in the schema's args description. Baseline 3 due to moderate coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the verb 'Run' and resource 'structured command array inside an open workspace'. The phrase 'real local execution' emphasizes it is not sandboxed. Distinct from sibling tools (file, task, etc.) which perform different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs alternatives. Does not mention prerequisites (e.g., workspace must be open) or conditions where a different tool is preferred. Only implies use when needing to execute a command.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• echoA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description does not contradict annotations (readOnlyHint: true). It adds the context that it's a smoke-test tool, implying no side effects. However, it doesn't provide additional behavioral details beyond what annotations already convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no unnecessary words. It efficiently conveys the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple echo tool with output schema and annotations, the description is mostly complete. However, it lacks guidance on usage or behavioral nuances, but given the tool's simplicity, it is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with a description for the 'message' parameter. The description adds no extra meaning beyond 'Message to echo.' Baseline 3 is appropriate as schema already documents the parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it's a 'Development smoke-test tool', indicating its purpose for testing. It is short but effectively communicates what the tool does. While it doesn't differentiate from siblings, the siblings are diverse, so confusion is unlikely.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies it's for testing purposes only (smoke-test), but does not explicitly state when to use it versus alternatives or provide exclusions. Usage context is minimal.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• workspace_openA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds little beyond the readOnlyHint annotation. 'Open' implies a read operation, consistent with the annotation, but it does not clarify what 'open' entails (e.g., loading config, validating path) or mention any side effects or permissions needed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no extraneous words. It efficiently communicates the core action and result.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple parameters, full schema coverage, and presence of an output schema, the description is mostly complete. It could explicitly state that the workspaceId is for subsequent operations, but this is minor.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for both parameters. The tool description does not add meaning beyond the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Open') and resource ('allowed local checkout workspace') and mentions the return value ('workspaceId'). It distinguishes from sibling tools like workspace_status and workspaceguard_info by focusing on opening a workspace for checkout.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. It does not specify prerequisites, when to use workspace_open vs. workspace_status, or scenarios where opening a workspace is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• git_statusA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, covering safety. The description adds exact command behavior. However, it lacks detail on potential output size or failure modes like non-git directories.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that conveys the essential action without unnecessary words. Perfectly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given an output schema exists, return value details are not needed. However, the description omits prerequisites (e.g., workspace must be opened and contain a git repo) and potential error conditions, leaving gaps for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must clarify the workspaceId parameter. The phrase 'inside an open workspace' hints at it, but does not explicitly state that workspaceId refers to an opened workspace or provide format constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Run git status --porcelain=v1') and the resource ('inside an open workspace'). It is specific enough to distinguish from sibling tools like git_diff and workspace_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a precondition (workspace must be open) but does not explicitly mention when to use vs alternatives or provide exclusions. The context is clear enough for a simple tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• policy_describeA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and the description 'Show' is consistent. No additional behavioral context (e.g., cost, caching, or side effects) is provided beyond what annotations offer. For a read-only tool, this is minimally adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with the verb and resource. No wasted words. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool is a simple read operation with no parameters and an output schema exists, the description fully covers the necessary context for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters are defined, so the description cannot add parameter meaning. Schema coverage is 100% trivially. Per evaluation rules, when there are 0 parameters, baseline is 4. Description is sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Describes clearly that it shows the current local policy profile, using the specific verb 'Show' and resource 'policy profile'. No ambiguous terms, and no sibling tools overlap in purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Does not explicitly state when to use this tool versus alternatives or when not to use it. However, given it's a simple query with no parameters and no sibling tools for policies, the intended usage is implied but not guided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• file_editA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool as destructive (destructiveHint=true) and not idempotent. The description adds value by clarifying the constraint of exact-match and uniqueness, though it doesn't disclose error handling for missing or duplicate text, or file accessibility.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, well-structured sentence conveys all essential information without redundancy. Every word is meaningful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple find-and-replace tool with an output schema and clear annotations, the description is largely complete. It could add edge-case behavior (e.g., error if not exactly one occurrence) but is adequate for most use cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description partially compensates by explaining oldText and newText as 'exact text block' and replacement. However, workspaceId and path are not explained, leaving ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the action (Replace), the target (exact text block), and a critical constraint (appears exactly once). It clearly distinguishes from sibling tools like file_write (overwrite entire file) and file_read (no modification).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use: when replacing a unique text block. The 'exactly once' constraint provides context for proper usage, but no explicit when-not-to-use or alternatives are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• task_statusA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint annotation, the description adds the important context that tasks are in-memory and explains the conditional behavior based on the presence of taskId. This adds value without contradicting annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences efficiently convey the core functionality with no unnecessary words. The action verb 'Return' is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema, the description doesn't need to detail return values. It covers the two modes of operation and the in-memory nature. It lacks error handling information but is adequate for a simple read operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates by explaining the parameter's effect: providing taskId returns that task, omitting it returns all tasks. This adds meaning beyond the schema's type-only definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a task by ID or all tasks, with a specific verb and resource. It distinguishes from sibling tools like task_start and task_update which are for other operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use each behavior (with or without taskId) but does not explicitly compare to sibling tools or provide guidance on when to use this tool vs others for similar purposes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.