Server Quality Checklist

Tool Scores

• skim_expandA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It only states it returns exact verbatim lines and that ranges work for skeleton lines. It does not disclose error behavior, prerequisites beyond a handle, or idempotency. Minimal behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first states the core purpose, the second adds details on input types. Every sentence is necessary, no fluff. Front-loaded with key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (2 string parameters, no output schema). The description covers what the tool returns and acceptable inputs, but lacks details on handle validity, error handling, or ordering. It is adequate but not fully complete for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description explains the 'anchors' parameter can be anchor ids or literal line ranges, adding meaning beyond the schema. However, the 'handle' parameter is not described (only implied from context), and schema coverage is 0%, so the description partially compensates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns exact verbatim source lines for anchor ids from a skim_open result. It specifies the verb 'return' and resource 'source lines', and distinguishes from siblings by mentioning skim_open as the source.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool should be used after skim_open by referencing 'from a skim_open result', and explains acceptable input types (anchor ids or line ranges). However, it does not explicitly state when not to use it or mention alternatives like skim_search.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• skim_searchA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Mentions case-insensitive search and efficiency, but omits details like result format (e.g., line numbers) or max_results behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, 10 words, front-loaded with purpose. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, yet description does not indicate what results contain (e.g., matched lines, positions). Missing guidance on interpreting output, essential for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description only adds value for 'query' parameter. 'handle' and 'max_results' are not explained, leaving significant gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the verb 'find' and the resource 'anchors/lines' with the specific behavior 'case-insensitive' and efficiency 'without reading the whole file'. Distinguishes from sibling tools like skim_open and skim_expand.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implicitly suggests use for quick searches without reading full file, but lacks explicit when-to-use or when-not-to-use statements. Sibling names provide some context for alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• skim_repoA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool returns skeletons (bodies folded), ranks by query or size, trims to budget_tokens, and lists excluded files. This gives a good understanding of behavior without contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, about four sentences, with the main purpose front-loaded. Each sentence adds meaningful information without fluff, earning its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and 0% schema coverage, the description provides a decent overview but lacks details on supported file types, error handling, or output format. It is minimally viable but has gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain parameters. It mentions query and budget_tokens implicitly but does not define them clearly. 'max_files' is not mentioned at all. Path is implied but not explained. The description adds minimal value beyond the schema structure.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool builds a lossless, ranked, token-budgeted map of a repo/directory. It specifies the output (skeleton of files, handles for expansion) and distinguishes from siblings like skim_expand and skim_open.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises using this tool to understand a codebase cheaply, then expand with skim_expand. It mentions using skim_open for files not included, providing clear context for when to use this tool vs alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• skim_runA
  4.1/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully explains behavioral traits: output is compact, deduped, nothing lost, returns exit_code and token report, and runs on the user's machine. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is fairly concise but could be tighter. It has two paragraphs with useful information, though some redundancy exists (e.g., 'compact' repeated).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description covers the main purpose and behavior but lacks parameter details and output structure beyond exit_code and token report. Error handling or security considerations are omitted.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%; the description does not add meaning to the parameters. It mentions 'command' but lacks details on format or timeout, leaving the agent uninformed about parameter specifics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs a shell command and returns a compact, expandable view. It distinguishes from siblings like skim_expand, which is mentioned for expanding output.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives guidance for verbose commands (test runs, builds, npm/pip, big logs) and mentions skim_expand for expansion, but does not explicitly state when not to use this tool or provide alternative tools for non-verbose cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• skim_patchA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses drift detection, newline preservation, re-skimming verification, fresh handle return, and the disable flag. This is comprehensive for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise yet thorough. First sentence immediately states purpose, followed by workflow, conditions, and behavioral details. No superfluous sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and moderate complexity, description covers error conditions (drift), side effects (newline preservation, re-skimming), and returned handle. Also mentions the environment variable for disabling. Very complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must compensate. It partially explains 'anchor' as 'anchored span (or literal range)', but handle and new_text are not explicitly described beyond context. Could be improved with direct parameter explanations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description explicitly states the action: 'Replace exactly one anchored span (or literal range) of a skimmed file on disk.' It clearly distinguishes from sibling tools like skim_open, skim_expand, etc., by specifying it is for patching a specific span after expansion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit workflow guidance: 'Expand the span first, edit that exact text, then patch.' Also explains when the tool refuses (drift) and how to handle it ('re-open and rebuild the patch'), plus the environment variable to disable it for read-only mounts.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• skim_openA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description fully discloses behavior: returns handle, report (token counts + compression ratio), skeleton, anchors; notes collapsed bodies not in skeleton; explains query usage for matches.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise two-paragraph description with front-loaded purpose. Every sentence adds value: purpose, return values, usage advice, collapsed bodies clarification, query optimization. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 2 parameters, no output schema, and no annotations, description is highly complete: covers return values, behavior, usage, query optimization, and distinguishes from siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but description adds meaning: path is file path, query is optional to get matches in same call. Could be more explicit but sufficient for agent understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it opens a large file and returns a compact skeleton with expandable anchor ids, not full text. Distinguishes from sibling tools like skim_expand (expand anchors) and skim_search (search).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to use instead of reading whole large file when only parts needed. Suggests passing query to avoid extra round-trip to skim_search. Provides clear when-to-use and when-not-to guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.