Server Quality Checklist

Tool Scores

• signet_signA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must cover behavioral traits. It mentions key sources but omits edge cases (e.g., both env and arg provided), error behavior, and the receipt format, leaving significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with verb and resource, no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 6 parameters and no output schema, the description explains the signing action but omits the return value (cryptographic receipt structure), which is needed for a complete understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (baseline 3). The description adds value by explaining the secret_key's optionality via env var fallback, which is not fully captured in the schema description alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Sign' and resource 'action (tool call)' with Ed25519 key, clearly distinguishing it from sibling tools like verification or key generation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains two key sourcing methods (env var vs argument) but lacks explicit guidance on when to use signing versus other tools or prerequisites like key availability.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• signet_content_hashA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It transparently states the computation (SHA-256 hash) and the canonicalization method (RFC 8785 JCS). It discloses no side effects or permissions needed, which is acceptable for a pure computation tool. However, it could mention that the output is a hex-encoded string.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, consisting of two sentences. It front-loads the action ('Compute SHA-256 hash') and the specification ('RFC 8785 JCS'). Every word contributes meaningfully without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (1 parameter, no nested objects, no output schema), the description is mostly complete. It covers the input and the core operation. However, it could mention that the return value is a hex-encoded hash string, as this is not obvious from the description alone.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, so the baseline is 3. The description adds 'any JSON value' which is already in the schema's parameter description. No additional semantics are provided beyond what the schema already conveys.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Compute SHA-256 hash of canonical JSON (RFC 8785 JCS). Accepts any JSON value.' It specifies the algorithm (SHA-256), the canonicalization standard (JCS), and the accepted input types. This distinctly sets it apart from sibling tools like signet_sign and signet_verify.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  While the tool's purpose is clear, the description does not provide guidance on when to use this tool versus alternatives. It does not mention prerequisites, exclusions, or contexts where hashing is preferred over signing/verification. Usage is implied but not explicitly guided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• signet_verifyA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite lacking annotations, the description discloses the return format and key format flexibility, providing sufficient behavioral context for a read-only verification operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is succinct, with two sentences that front-load the primary purpose and add key detail without unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple verification tool with no output schema, the description covers the return structure and key parameter details, making it sufficiently complete for agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for both parameters, and the description adds value by explaining the accepted public key formats (bare base64 or ed25519:base64), enriching the schema's simple type definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool verifies a Signet receipt signature, explicitly lists the return value as {valid: true/false}, and specifies accepted key formats, distinguishing it from sibling tools that deal with content hashing, key generation, and signing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for verifying signatures but does not provide explicit guidance on when to use this tool vs siblings, nor does it mention prerequisites or error conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• signet_generate_keypairA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that the secret key is not returned and points to CLI for secure management, providing critical behavioral context beyond the schema. No annotations present, so description carries full burden.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no redundancy, front-loading the key action and outcome.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose and output limitation. Lacks detail on public key format or usage, but adequate for a simple key generation tool with no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist in schema; description adds no parameter info but baseline is 4 for zero-parameter tools. Sufficient for the tool's simplicity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'Generate a new Ed25519 keypair' and specifies it returns only the public key, distinguishing it from siblings like sign, verify, and content_hash.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises using Signet CLI for secret key management, giving implicit guidance on when not to use this tool. Lacks explicit alternatives but context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.