Server Quality Checklist

Tool Scores

• auth_statusA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It thoroughly discloses that the tool is read-only, idempotent, safe to repeat, makes a lightweight verification call, never writes to the audit log, and never mutates credential storage. It even details the error behavior for unsupported platforms. This is exemplary transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-organized into clear segments: purpose, usage, behavioral disclaimers, parameters, examples, and return format. It contains minimal redundancy, though 'read-only' and 'never writes to the audit log' are repeated across sentences. Still, every paragraph adds needed nuance, so it's appropriately sized for a tool with no annotations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has one parameter, no annotations, and no schema-level parameter docs, the description covers all necessary aspects: what the tool does, when to invoke it, its exact behavioral constraints, accepted parameter values, example calls, and the detailed return shape. Even though an output schema exists, the description goes beyond it by explaining success/failure semantics. Nothing critical is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema merely defines `platform` as a string with 0% description coverage. The description compensates fully by specifying allowed values ('reddit' or 'youtube'), the structured error for other values, and example calls. This is exactly the kind of added meaning needed for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Read-only check of whether BYOK credentials are already stored and still valid for a platform.' It clearly distinguishes this from siblings by noting it should be called before `search` and that it does not touch the search API or audit log. The tool's limited scope (verification only) is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use guidance: 'Call this before `search` to confirm credentials are configured and working, or whenever an agent needs to report connection health.' It also explains what it is not for (setting/clearing credentials) and contrasts with remote mutability. This is direct, actionable context for an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full weight and discloses critical side effects: makes a live network call consuming rate limits, appends a tamper-evident hash-chained entry to the audit log on success, is not read-only, and returns an error string rather than throwing an exception. This is thorough behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Though lengthy, the description is well-structured: purpose/constraints, usage guidelines, side effects, parameter details, examples, and return format. Every sentence adds necessary information; there is no redundancy or filler, making the length appropriate for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 9 parameters, no annotations, and an output schema is referenced but not detailed, the description covers all necessary context: prerequisite credentials, platform-specific restrictions, failure modes (missing credentials, unsupported platform), side effects, return shape, and pagination. It is fully complete for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage and only parameter titles, but the description fully compensates by explaining each parameter's purpose, platform-specific applicability (e.g., subreddit for reddit only, channel for youtube only), defaults, caps (max_results capped at 100 for reddit and 50 for youtube), and even example calls. This adds substantial meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Search Reddit or YouTube through its official, documented API... and return matching posts/videos as structured JSON.' It also distinguishes itself from sibling tools by explicitly directing credential checks to auth_status, making the purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit when-to-use ('Call this whenever an agent needs recent or historical public content... for research, monitoring, or compliance review') and when-not-to-use (platforms other than reddit/youtube, credential checks). It names auth_status as the cheaper alternative for credential validation, satisfying the need for alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_logA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses read-only behavior ('opens and reads the log file but never writes'), idempotency, the prerequisite of an existing log file, and the structured failure mode instead of raising an exception. It also explains return values on both success and failure, leaving no behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is detailed but tightly organized, moving from purpose to usage, safety, prerequisite, parameter format, and return value. No sentence is redundant; each adds necessary information for correct invocation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers all relevant aspects: what the tool verifies, when to call it, read-only guarantees, prerequisites, parameter semantics, and return values including failure cases. It is self-sufficient for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Though the schema only shows an optional path with a default of null, the description explains the default value (`./auditreach.log.jsonl`), gives example calls, and clarifies that the path points to the log file. This adds essential meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Verify that the local hash-chained audit log has not been tampered with' and explains the method (walking entries and confirming hash chains). This clearly distinguishes it from siblings like search and auth_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to call — 'before sharing the log with a third party, after any manual edit to the log file, or periodically as a trust check' — and clarifies when not needed: 'it does not need to run before every search'. This provides clear usage boundaries relative to the search sibling.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.