Server Quality Checklist

Tool Scores

• scan_fileB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The description doesn't state whether the scan is read-only, whether any side effects occur, what the return format looks like, or any ownership/permission requirements. For a tool with zero annotations, this lacks adequate transparency about what happens during execution.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact—a one-sentence summary plus a single documented parameter with its arg meaning. It is appropriately brief with no filler. Minor deduction for the Args section being a lightweight docstring format rather than a structured rich description, but overall it earns its sentences well.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 1 parameter, no output schema, and no annotations, the tool is relatively simple. The description covers the purpose and the single parameter adequately, but lacks detail on what the scan result looks like (e.g., return format, whether it returns findings or just a pass/fail) and no behavioral context. It's adequate for a minimal tool but leaves the agent guessing about output structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  While schema description coverage is 0%, the description does add value by explaining file_path is 'Absolute path to the Python file,' adding format (absolute) and type (Python) constraints beyond the schema's bare 'string' type. With only one parameter, this adds sufficient semantic meaning, though the name/schema combination was already fairly clear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource ('Scan a file') and clarifies the domain (potential hallucinations) with concrete examples: undefined symbols, missing imports, incorrect API calls. It clearly states what the tool does, though it doesn't need sibling differentiation as no siblings exist.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. There's no mention of language scope beyond 'Python file' in the arg description, no prerequisites (e.g., file must exist), no context on when scanning is appropriate. The only implicit context is scanning for hallucination-type issues, which is weak guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.