Server Quality Checklist

Tool Scores

• build_import_graphA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals that nodes are auto-collapsed for large repos, edges are directed, and the result is JSON. These details go beyond a trivial statement and help the agent anticipate output structure. However, it does not mention whether the tool is read-only, performance implications, or side effects, but the nature of the task strongly implies a non-destructive analysis.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is impressively concise, using two sentences to cover the core action, supported languages, node/edge behavior, auto-collapse rule, and return type. Every sentence adds value with no redundancy or fluff, and the main verb 'Extract' is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of an output schema, the description adequately explains the return value (JSON), node/edge representation, and the autoregressive collapsing behavior for large repos. It also specifies supported languages, which is a critical constraint. It could go further by describing an example output structure or handling of unsupported languages, but overall it is complete enough for a one-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers the single parameter (path) with a clear description of 'Absolute or relative path to the repository root' (100% coverage). The tool description does not add extra parameter semantics, so the baseline of 3 is appropriate since the schema already provides sufficient meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: extract intra-repo import/require relationships for JavaScript/TypeScript and Python. It specifies the resource (repo), the exact type of relationships (directed imports), and the output (JSON). It distinguishes itself from sibling tools like map_repo or generate_diagram by focusing on import graph extraction rather than generic mapping or visualization.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus alternatives like map_repo, scan_repo, or generate_diagram. It implies usage for import analysis but doesn't explain exclusions or mention that for diagram generation one should use generate_diagram. An agent is left to infer the use case, which is a clear gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_repoA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses determinism, output format (JSON facts only), and the scope of extraction, but does not explicitly mention side effects or read-only nature. This provides moderate transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the action and output, no redundant phrases. Every sentence contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple single-parameter schema and lack of output schema, the description provides a reasonable overview of what will be returned, listing the categories of facts. It is sufficient for an agent to anticipate the tool's behavior, though exact JSON structure isn't specified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description does not add information about the 'path' parameter beyond the schema's description. With 100% schema coverage, the baseline applies; the description neither clarifies nor contradicts parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with specific verbs ('extract facts') and lists the exact resource types covered. It distinguishes from siblings by emphasizing determinism and 'no interpretation', which differentiates it from map_repo or diagram tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a factual inventory of a repository is needed, but does not explicitly state when to avoid this tool or name alternative tools. The 'no interpretation' clause gives a subtle hint but no explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• render_diagramA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the HTML uses CDN (Mermaid or Viz), that it writes a raw source sibling, and that it returns absolute paths. This covers file-writing side effects and dependency on internet/CDN, which is valuable. It stops short of mentioning error handling or directory creation, but the key behavioral traits are disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence with important information front-loaded ('Write a self-contained, shareable HTML page'). It avoids redundancy with the schema and earns its place by clarifying output artifacts and return value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters, no annotations, and no output schema, the description covers the core purpose, output files, and return format. It does not explicitly differentiate from sibling tools or describe error behavior, but for its complexity it is sufficiently complete to guide an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline of 3 applies. The description adds no extra parameter details beyond the schema; it repeats the raw source and path output. However, it reinforces the relationship between format and source, but does not compensate beyond what schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool writes a self-contained HTML page and a raw source file. It uses the specific verb 'Write' and identifies the resources (HTML, .mermaid/.dot), which distinguishes it from sibling tools like generate_diagram that likely create diagram source rather than rendering it to shareable HTML.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: when you have Mermaid/DOT source and want a shareable HTML page. However, it does not explicitly state when to use this tool versus alternatives like generate_diagram, nor does it provide exclusions or prerequisites. The context is present but not overtly comparative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_diagramA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It adds behavioral context beyond the schema by noting the output is a 'draft the calling model is expected to refine,' implying the result is not final and may need iteration. It also explains the level parameter's effect on granularity, which is not in the schema. It does not mention side effects, but generating a string implies a read-only operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, with the first sentence front-loading the core action and output. Every sentence adds value: purpose, level semantics, and the draft-refinement caveat. There is no redundancy or filler, making it concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and no annotations, the description covers the essential aspects: purpose, parameter semantics, output nature (draft diagram string), and context (combines scan/import facts). It could explicitly note prerequisites (e.g., that scan_repo and build_import_graph should be run first), but the phrasing 'combine scan + import facts' implies this dependency. Overall, it is sufficiently complete for an agent to understand invocation and result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% parameter coverage, providing descriptions for path, level, and format. The description adds value beyond the schema by elaborating on the level values: 'high' shows modules/directories and 'detail' shows files grouped by module. It also confirms the default formats inline, though these are already in the schema. The level semantics are the key addition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb phrase 'Combine scan + import facts into a DRAFT diagram string,' clearly defining the tool's purpose and output. It distinguishes itself from siblings like render_diagram by explicitly labeling the output as a draft for refinement, and names the supported formats (Mermaid/DOT), making the tool's role unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: after scan and import facts are available, to produce a draft diagram. It does not explicitly name alternatives or exclusions (e.g., 'use render_diagram for final output'), but the 'draft' framing and sibling names imply the appropriate workflow. This is clear context but lacks explicit when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• map_repoA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does well: it discloses internal orchestration ('runs scan + import-graph + render internally'), side effects (writes architecture.html and raw source into the repo), configurable output location (outPath), and the return payload (HTML path, facts summary, draft diagram source). It lacks potential permission or idempotency details but gives substantial behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no fluff: the first states the primary action and return values, the second covers defaults and alternative tool usage. All information is front-loaded and purposeful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a medium-complexity orchestration tool with no output schema or annotations, the description covers what it does, what it returns, filesystem side effects, and how it relates to sibling tools. The description is complete enough for an agent to invoke it correctly and understand high-level consequences.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already has 100% coverage, so baseline is 3. The description adds value by explaining the default output ('architecture.html plus the raw source (.mermaid or .dot) into the repo') and clarifying the outPath behavior ('write elsewhere'). It also reinforces the format and level concepts, going beyond the schema's basic descriptors.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Point at a folder and get a downloadable architecture diagram in one call' and clearly names the resource and outcome. It also explicitly distinguishes the tool from its granular siblings by positioning it as a one-shot orchestrator.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Exactly states when to use the tool ('Point at a folder and get...') and when not to ('Use the granular tools... only when you need to compose the steps yourself'). It names all alternative sibling tools, providing clear direction for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.