Server Quality Checklist

Tool Scores

• redact_pdfA
  3.9/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses important behaviors: permanent deletion, irreversibility, OCR handling, PII detection language coverage, and EU/Swiss infrastructure. However, the annotations declare destructiveHint=false, which directly contradicts the description's 'permanent, irreversible PDF redaction' and 'ephemeral deletes the original after processing.' This is an annotation contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no filler: the first front-loads the action and non-blocking behavior, the second gives critical permanence and capability context, and the third provides usage routing and next steps. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema, the description states the return contract ('Returns a job_id plus one document id per file') and the required next steps. Combined with the 100% schema coverage and clear sibling relationships, the agent has enough context to call and continue the workflow correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the schema itself has rich descriptions for each parameter, including default behavior for pii_categories and idempotency_key semantics. The tool description adds little parameter-specific meaning beyond the schema, so it stays at the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Start a redaction job and return immediately without waiting.' It also differentiates itself from the sibling tool redact_pdf_and_wait by emphasizing the non-blocking behavior and the permanent, irreversible nature of the redaction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to prefer redact_pdf_and_wait unless the user needs parallel processing or a large document, and it names the follow-up tools (get_job_status, download_redacted). This gives the agent actionable routing guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• download_redactedA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds meaningful behavioral context beyond the annotations: outputs are ephemeral and deleted after a short window, the original file is never modified, and the default output path is derived from the input path. These details help the agent anticipate side effects without contradicting the destructiveHint annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the core purpose, followed by the critical conditions and retention warning. Every sentence earns its place, and there is no redundant fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers prerequisites, retention timing, output path defaults, and a safety guarantee about the original file. It does not describe the response/return value, but for a download operation this is a minor gap given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters. The description repeats some parameter context, such as where the document id comes from and the default output path, but does not add substantially new parameter-level meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies a concrete verb and resource: downloading the finished redacted PDF for one document. It also identifies the source of the document id, which distinguishes it from sibling tools like redact_pdf and get_job_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear condition for use: the document must report status "redacted" and the download must happen before the ephemeral retention window closes. It does not explicitly discuss when to prefer a sibling tool, but the usage context is specific enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_account_statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint annotation already communicates that this is a safe read operation. The description adds non-obvious context about account plan quota and credit packs, and explains that 'out of pages' failures are permanent quota problems rather than transient errors. This is useful behavioral context beyond the annotations, though it does not describe the exact return payload.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concisely written and front-loads the core purpose before the recommended usage timing. The second paragraph about redaction billing and quota errors is somewhat tangential to what this tool itself does, but it is still relevant context and does not make the description bloated.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, zero parameters, and no output schema, the description covers the essential what, when, and error-handling context. It does not enumerate the specific fields returned in the account status response, but the stated purpose ('see which account it belongs to') provides enough for an agent to invoke the tool correctly for its intended use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there are no parameter semantics for the description to clarify. According to the baseline for zero-parameter tools, a score of 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('check') and resource ('API key and account'), and clearly conveys the tool's diagnostic purpose. It is easily distinguishable from the sibling redaction and job-status tools because it is explicitly framed as a pre-batch validation step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: call this before a large batch to fail fast on a bad or missing key rather than after uploading. It also provides an exclusion for retrying 'out of pages' errors, which is a clear error-handling instruction that helps the agent decide when not to retry a related call.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_job_statusA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes well beyond the annotations by explaining the full status progression, which statuses are terminal, and the recommended polling behavior. It also discloses that each document reports its own status, page_count, and error_message, giving the agent a realistic model of the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed only by high-value behavioral details: statuses, final states, polling advice, and per-document reporting. Every sentence contributes useful information without repetition or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter status-check tool with no output schema, the description is remarkably complete. It explains the status model, terminal states, polling cadence, and the per-document outcome fields, so an agent has enough context to call the tool and interpret results correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes job_id as 'The job_id returned by redact_pdf,' so coverage is complete. The description does not need to add parameter-level detail and does not repeat it; this is an acceptable baseline when the schema carries the meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Check a redaction job's progress.' It clearly identifies the tool's function and distinguishes it from sibling tools like redact_pdf, redact_pdf_and_wait, and download_redacted, which perform different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use the tool: to check whether a job is still in flight and what statuses mean. It also provides polling guidance, telling the agent to wait seconds between checks rather than tight-looping. It does not explicitly name alternatives, but the sibling list and the status lifecycle make the intended use unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• redact_pdf_and_waitA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate destructive behavior, and the description strongly reinforces it by stating redaction is 'permanent, irreversible' and that text is deleted, not hidden. It also adds non-obvious behavior: OCR on scanned PDFs, 100+ languages, blocking for 10-60 seconds, server-side continuation on timeout, and EU/Swiss infrastructure. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core behavior and 'START HERE' marker, then covers use cases, timeout behavior, and API key requirements in tight paragraphs. It is slightly longer than the minimum needed, but every section earns its place and the structure makes it scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description compensates by stating the return values (redacted PDF, job_id on timeout), explaining blocking behavior, mentioning prerequisites (API key), and directing to sibling tools for follow-up. For an 8-parameter, destructive, blocking tool this is fully adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the parameter descriptions already explain semantics well, including guidance like 'do not pass an empty list' for pii_categories and defaults for output_path. The main description adds no significant parameter-level meaning beyond what the schema provides, so a baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Redact'), a clear resource ('PDF'), and explains the full one-call flow: upload, detect, redact, wait, return. 'START HERE' and 'one-call tool' explicitly differentiate it from the multi-step sibling tools like redact_pdf and get_job_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It states when to use the tool ('whenever someone wants PII... removed from a PDF'), gives concrete alternatives for edge cases (poll get_job_status instead of re-uploading on timeout, call try_demo if no API key), and frames it as the starting point. This is explicit usage guidance with named fallback tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• try_demoA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint and openWorldHint, but the description adds meaningful behavioral context beyond the schema: it requires no API key or upload, never touches user data, costs nothing, and returns a real redaction with detected PII and a link to the redacted PDF. This fully discloses side effects (none) and output characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short paragraphs with zero waste. The core purpose is front-loaded in the first sentence, followed by output details and usage guidance. The exclusion and sibling reference are efficiently placed in the final sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter demo utility with no output schema, the description covers everything an agent needs: what it does, what it returns, when to use it, and what it cannot do. It also names the appropriate sibling for real documents. No meaningful gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and 100% schema coverage, so the baseline is 4. The description reinforces this by stating 'no API key and no upload', but there are no parameter semantics to add beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific action and resource: 'Verify this server works... Runs the keyless demo endpoint, which returns a real redaction of a built-in synthetic-PII sample.' It clearly distinguishes the tool from siblings by explicitly stating 'It cannot redact a real document — use redact_pdf_and_wait for that.' The purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: 'Use this to confirm connectivity or to show a user what redaction output looks like before they sign up.' It also names the alternative tool for real redaction and states the exclusion: 'It cannot redact a real document — use redact_pdf_and_wait for that.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.