Server Quality Checklist

Tool Scores

• list_formatsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only, idempotent, non-destructive behavior. The description adds valuable context about the exact output content (drivers, extensions, support limitations), which goes beyond the annotations. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, with the primary purpose and scope front-loaded. It avoids redundancy and every sentence earns its place—the first states what it lists, the second when to call it. No unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple, parameterless list tool with no output schema, the description fully explains what the tool returns (drivers, extensions, and support limits). There is nothing missing for an agent to correctly invoke it and interpret the result. Complexity is low, so this is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema is fully covered (100% by default). There is nothing to add about parameters; the description doesn't need to explain any. The baseline for zero parameters is 4, and no additional info is required.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('List'), a clear resource ('document formats DeckProbe can inspect'), and specifies the content (drivers, extensions, and where support stops). It effectively distinguishes itself from sibling tools like probe and list_targets by focusing on format capabilities rather than probing or target listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: 'Call this when you are unsure whether a file type is supported at all.' It doesn't mention alternatives, but the trigger condition is clear and implies that if you have a specific file, you would use probe instead. It could be improved by naming the sibling tools explicitly, but the guidance is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_targetsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, openWorldHint=false, and destructiveHint=false, covering safety. The description adds behavioral context by spelling out returned fields and the probe rejection behavior, which matters for planning calls. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences front-load the core purpose and output contents, then give one practical usage rule. No filler or repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With a rich annotation set, fully documented parameters, and a description that names the output fields and the prerequisite call, an agent has enough to invoke the tool correctly even with no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already explains format and detail fully, including enumerations and examples. The description adds no new parameter-level semantics beyond naming the report contents, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb (list) and resource (targets for a given format), enumerates exactly what is included (aliases, value type, minimum probe level, cost class, selector membership), and contrasts with probe by stating probe rejects unknown targets. This distinguishes it from sibling tools list_formats and probe.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly instructs when to call: before naming a target not already seen, because probe rejects unknowns rather than guessing. It also ties the format parameter to list_formats, implying the prerequisite and differentiating from list_formats.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• probeA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, so the safety profile is covered. The description adds critical behavioral context beyond those hints: it does not extract, render, run macros, follow external references, or open a network connection; it handles legacy formats; it can hit resource limits (referenced by 'Raise after a BUDGET_EXCEEDED error'). This is exactly the kind of non-obvious behavior an agent needs to trust the tool with untrusted files.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: opening scope sentence, format list, usage hint, and a final 'does not' sentence. It is front-loaded with the core purpose and scoping. It earns its sentences; only a minor redundancy exists (the 'Prefer this over...' sentence partially repeats the 'does not' list). Not a 5, but far above the typical terse definition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 7 parameters, nested budget object, output schema present, and complex behavior (format drivers, confidence levels, presets, resource limits), the description provides substantial orientation: preset suggestions, the @summary caveat, the budget-error hint, and a clear non-extraction guarantee. The description does not explain what each target means, but it correctly defers to list_targets. So it's complete enough without being exhaustive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 7 parameters well. The description adds important cross-references that the schema alone does not capture: the distinction between @summary and full-read statistics, the warning that @summary omits page_count so ask explicitly, and that list_targets should be used rather than guessing names. This goes beyond a baseline 3 by explaining how the parameters interact with presets and the driver's default set.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Read facts about one local PDF, Microsoft Office, or Apple iWork document') plus a precise resource list (page/slide/sheet counts, title/author/dates, encryption, macros, structure, assets, integrity). The scope is explicit: it reports facts ABOUT the document, never extracts text, renders, runs macros, or opens connections. It also names sibling tools (list_targets, probe_batch) and distinguishes from unzipping/byte-parsing by hand. This is clear and differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises 'Start with targets ["@summary"], or ["@security"] to triage an untrusted file.' and 'Prefer this over unzipping the document or parsing its bytes by hand.' It also warns about @summary omitting statistics and instructs to 'Call list_targets rather than guessing a name.' This provides direct when-to-use and when-not-to-use guidance, plus pointers to alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• probe_batchA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds substantial behavioral detail beyond the annotations: one engine process for the batch, per-path results in order, error envelopes per file, failure isolation, default view and when to switch. Annotations already cover read-only/idempotent/destructive, and the description does not contradict them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, zero fluff. The first sentence front-loads purpose and use cases; the second covers path handling and return behavior; the third gives view guidance. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 6 parameters, nested objects, and a full output schema, the description covers the essential usage context: batch behavior, order of results, error isolation, and view default. It does not mention budget override or target selection, but those are adequately documented in the schema. The description is sufficient for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds valuable semantic guidance: 'Takes literal paths; expand any glob yourself first' clarifies the paths parameter, and the view default rationale ('inventory rarely needs per-target evidence') adds context beyond the schema. It does not elaborate on every parameter, but the key ones receive useful extra meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool probes many local documents in one call, listing concrete use cases (inventory a folder, triage a batch, compare files). It distinguishes itself from the sibling 'probe' by explicitly being the batch counterpart.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly explains when to use this tool ('Probe many local documents in one call'), gives practical scenarios, and contrasts cost efficiency ('much cheaper than calling probe once per file'). It also gives guidance on view selection ('pass view: report when it does').

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.