Server Quality Checklist

Tool Scores

• scan_statusA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does well: it discloses possible statuses, what happens when done (includes headline result fields and skipped sources due to missing API tokens), and implies it's a read-only status check. It does not mention error/timeout behavior, so not a 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: the first clause states the core purpose, followed by concise status values and a clear 'when done' detail plus a direct next-step instruction. Every sentence earns its place with no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a simple one-parameter tool with no output schema, the description covers the essential behavior: status outcomes, key result fields when complete, skipped sources, and recommended follow-up. It lacks a full return schema and edge-case behavior, but is reasonably complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides only a bare 'scan_id' string with 0% coverage. The description adds context that scan_id refers to a scan initiated by scan_target, but does not clarify format, required source, or how to obtain it beyond implication. This partially compensates for the schema gap but is not fully descriptive.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Check a scan started by scan_target' with specific status outcomes (running, done, error). It distinguishes itself from siblings by focusing on status checking rather than starting (scan_target), listing (list_scans), reporting (get_report), or credential handling (reveal_credentials).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear sequencing context ('Check a scan started by scan_target' and 'Then call get_report(scan_id)'), indicating when to use this tool in a workflow. However, it does not explicitly name alternatives or state when not to use it, so it falls short of full exclusionary guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reveal_credentialsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses that the tool exposes the most sensitive data (plaintext passwords) and sets an explicit invocation gate, a critical behavioral trait. However, it does not discuss authorization, logging, or error behavior, so it is not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short lines with a clear front-loaded purpose followed by a critical gate note. Every sentence serves to clarify the tool's sensitive nature, with no redundant content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with an output schema, the description provides crucial usage context that helps an agent decide to invoke it. The main gap is the undocumented scan_id, but sibling tools likely provide this identifier. The sensitivity gate is well-covered, making it mostly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain scan_id beyond referencing 'a scan'. The name and context imply scan_id identifies the scan, but no format, source, or validation details are given. The description must compensate for the sparse schema and largely fails to do so.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Return' and specific resource 'leaked-credentials dossier', clarifying it provides plaintext passwords. It distinguishes from sibling tools by labeling it the explicit gate for the most sensitive output, separating it from get_report or list_scans.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use guidance: only call when the user clearly asked for actual leaked credentials. This implies when not to use it (e.g., if a user only wants a summary) and frames the tool as a gated operation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_reportA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosure. It clearly states that the markdown output omits the leaked-credentials section, which is a significant behavioral trait not inferable from the schema. It also notes the default format and the alternative for credentials. However, it doesn't mention error handling, permissions, or response structure beyond this.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with the core purpose, and every sentence adds critical information. No fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with two parameters and an output schema, the description covers the essential behavior, including format options and the credential removal caveat. It could benefit from a brief note on when to prefer this over scan_status/list_scans, but the output schema and sibling names fill some gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no descriptions for the parameters (0% coverage). The description adds meaning to fmt by listing valid values 'md' (default) and 'json', and implicitly explains scan_id through the 'scan's report' context. While scan_id is self-explanatory, the fmt clarification is valuable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: returning a scan's report. It distinguishes itself from siblings by focusing on report retrieval, and mentions the fmt parameter for format selection. The verb 'return' and resource 'scan's report' are specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by explaining the default format and the crucial caveat that leaked credentials are removed from markdown, directing users to reveal_credentials for that data. It doesn't explicitly contrast with scan_status or list_scans, but the purpose alone makes the distinction clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_scansA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description adequately discloses the tool's output characteristics (newest first, specific fields). It provides transparency about what the tool returns without any hidden side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that is direct and complete, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, and the description covers its key behavior and output ordering. With an output schema present, the description need not elaborate on return structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, and the description mentions the output fields, which is sufficient since there is nothing to explain about input semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists previously run scans with a specific order (newest first), which distinguishes it from sibling tools like scan_status or get_report that serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes it clear this is for retrieving past scans, providing a clear context for when to use it. It does not explicitly exclude other tools, but the context is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_targetA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden. It discloses that the scan runs in the background, takes several minutes, returns immediately with a scan_id, allows only one scan at a time, and that leaked credentials are not in the headline result. This is rich, non-obvious behavior beyond what structured fields could convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three tightly written sentences that deliver all essential information without padding. It front-loads the purpose and immediately follows with actionable workflow steps, constraints, and input requirements. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema and no annotations, the description covers the essential context: the asynchronous nature, the return value (scan_id), the polling workflow, the concurrency limit, input prerequisites, and where to find sensitive data. This is complete for an agent to correctly select and invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 6 optional parameters with no descriptions, and the description compensates for most: it names email, phone, name, city, and state in context, and specifies that at least one of email/phone/name is required, with name working best alongside city/state. However, zip_code is not mentioned at all, leaving a small gap in parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Start a privacy-OSINT scan,' clearly specifying both the action and the resource. It differentiates itself from sibling tools by mentioning the immediate return of a scan_id and directing users to subsequent tools (scan_status, get_report, reveal_credentials).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs when to use this tool ('Provide at least one of email / phone / name') and what to do next: 'Poll scan_status(scan_id) until it reports "done", then call get_report(scan_id).' It also states the concurrency limitation and points to reveal_credentials for leaked credentials, providing clear alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.