Server Quality Checklist

Tool Scores

• daily_statsC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description bears full responsibility for behavioral disclosure. It only states the function, without revealing whether this is a read-only operation, any required authentication or rate limits, or what happens if the tenant has no logs. This is minimal transparency for a stats endpoint.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence, which is appropriately sized. However, it is too sparse to be maximally effective. Every sentence should add value, and while this one does state the purpose, it lacks additional useful context. It is not verbose, but underinformativeness detracts from conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there is an output schema (though unknown content) and 0% parameter documentation, the description should provide more context, such as the time range of the stats, granularity (e.g., daily, hourly), or that it returns aggregated volume. The description leaves too many essential details unspecified for a stats tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, meaning the description must compensate for missing parameter documentation. The description mentions 'for a tenant' but does not explain what 'tenant_id' represents or how to obtain it. The 'api_key' parameter is not mentioned at all, leaving its purpose unclear. The description adds little value beyond the schema field names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly uses a specific verb 'return' and resource 'daily log volume stats' for a tenant. It effectively communicates what the tool does. However, it does not differentiate from sibling tools like 'log' or 'search', which might also return log-related data, so it lacks explicit sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites, limitations, or context. For example, it could clarify if this is for aggregated daily stats or real-time data, relative to 'search' which might provide detailed log entries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description adds some behavioral info: 'Returns newest-first' and implies idempotence (query). No information on rate limits, auth requirements, or side effects, but the read-only nature is inferred.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences that are front-loaded with key information. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description lacks detail on parameter usage, error cases, and behavior beyond ordering and empty filters. For a complex search tool with 7 parameters, this is insufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description does not explain any of the 7 parameters. The only hint is 'Empty filters' but no parameter-specific semantics are provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Query the log' and specifies behavior (empty filters match everything, returns newest-first). It distinguishes from siblings like 'log' by being a search tool, though explicit differentiation is lacking.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions 'Empty filters match everything' which guides usage. However, no context on when to use this versus alternative tools like 'log' or 'daily_stats', and no exclusions are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_chainB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the return value ('OK or first sequence number') but does not mention side effects, potential errors, or whether the operation is read-only. Moderate transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, each adding value: first states purpose, second states return value. No wasted words. Front-loaded effectively.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple verification tool with 2 parameters and an output schema, the description covers purpose and return value but lacks parameter explanations and usage context. It is minimally complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description adds no information about the parameters (tenant_id, api_key). It does not explain their purpose or format, leaving the agent with only the schema's titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Re-verify', the resource 'log chain', and the scope 'for a tenant'. It also specifies the return value, distinguishing it from sibling tools like 'log' or 'search'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for checking chain integrity but does not explicitly state when to use or when not, nor does it compare to alternatives. It's adequate for a tool with no similar siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sign_day_attestationA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the attestation includes a tip hash for non-retroactive modification proof. However, it lacks details on side effects, permission requirements, or rate limits. The output schema exists but the description doesn't leverage it to explain behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences and front-loaded with the primary purpose. It efficiently conveys restriction, use case, and a key detail. Minor length is acceptable given the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema and no annotation coverage, the description provides a high-level purpose but is incomplete for understanding parameters and full context. It doesn't cover what the output contains beyond the tip hash, or how to interpret the signed attestation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter meanings. It only implicitly refers to 'for a given day' (date_utc) and 'Pro/Enterprise only' (likely tenant-based). It does not explain tenant_id, api_key, email, or their roles, leaving a significant gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool emits a signed attestation for log integrity and volume, which is distinct from sibling tools like daily_stats or search. It specifies the resource (attestation for a day) and action (emit signed).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear use case: end-of-day evidence for auditors/Trust Center. It mentions it's Pro/Enterprise only, which helps restrict usage. However, it does not explicitly state when not to use this tool or compare to alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• logA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description notes that entries are 'signed, hash-chained' and mentions PII hiding for payload_hash, but does not cover authentication, rate limits, or error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: one introductory sentence and a clear bullet list of parameters with explanations, no extraneous content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 8 parameters (4 required) and no visible output schema, but the description covers purpose and most parameter semantics adequately, though it lacks details on prerequisites or return behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description explains 7 of 8 parameters with purpose (e.g., 'tenant_id: customer / organisation identifier'), but omits api_key.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Append' and the resource 'signed, hash-chained log entry', which distinguishes it from sibling tools like search or verify_chain.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when or when not to use this tool compared to siblings such as daily_stats or verify_chain; usage is only implied by the tool's purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.