Server Quality Checklist

Tool Scores

• market_stateA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds an important caveat that this is 'a model, not live telemetry,' which helps set expectations about data freshness and accuracy. However, with no annotations provided, it does not explicitly disclose side effects (or lack thereof) or other behavioral details such as latency or consistency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences, front-loaded with the core purpose and ending with a useful caveat. Every word earns its place; no unnecessary fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read-only snapshot, the description covers the main output areas and adds the 'model, not live telemetry' caveat. It could be slightly clearer about what the 'three numbers' are, but the title fills that gap, making it reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool accepts no parameters, so the baseline is 4. The description provides enough context about what the snapshot reports without needing parameter explanations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The title and description clearly indicate a read-only market state snapshot, naming specific data points (supply, allocated, utilized, matching/scheduling gaps). It is distinct from the simulation and proof request siblings, though it does not explicitly name an alternative for comparison.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given for when to use this tool versus the sibling tools. The description implies a status-check use case but does not state when to prefer it over simulate_coordination or poll_proof.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• poll_proofA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It reveals that the result carries a proof id and attestation once the request settles, and mentions an unusual delivery mechanism ('the way Fermah returns a proof to a Seeker callback URL'). However, it does not disclose whether the tool is read-only, what happens when the request is still pending, or any error/timeout behavior, leaving gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each earning its place: the first states the core action, the second elaborates on the result, and the third adds a meaningful qualifier about human involvement. It is concise with no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple single-parameter schema and no output schema, the description gives some context (result carries proof id and attestation) but lacks details on pending-state behavior, error handling, or a concrete return structure. The cryptic reference to 'the way Fermah returns a proof to a Seeker callback URL' adds ambiguity rather than clarity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides 100% coverage for the only parameter (requestId) with its description 'id returned by submit_proof_request'. The tool description adds no additional semantic detail about parameters, so per rubric the baseline of 3 applies when the schema handles the load.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb and resource: 'Check a Proof Request.' It distinguishes itself from siblings like submit_proof_request (which creates the request) and market_state (which likely checks broader market conditions) by focusing on a single proof request and its settlement outcome.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage after submitting a proof request, but does not explicitly say 'use this after submit_proof_request' or contrast with alternatives. The schema's parameter description ('id returned by submit_proof_request') hints at the dependency, yet the main description lacks overt when-to-use versus when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• simulate_coordinationA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the burden of behavioral disclosure. It reveals that the tool runs a full simulation and returns utilization and gaps, but does not state whether it is read-only, has side effects, or requires setup, which is a notable gap for a simulation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, with the core action front-loaded and no filler. The second sentence adds valuable context without bloat.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, the description mentions the return values but does not explain what 'the two gaps' are or disclose any side effects or prerequisites. This leaves ambiguity for an agent attempting to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 75% (seed lacks a description), and the description does not compensate for that gap. It adds context about supply being constant but does not detail individual parameter meanings beyond what the schema already provides, offering only marginal added value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs the proofmarket-sim engine over a full horizon for a coordination regime and returns utilization and two gaps. It distinguishes itself from operational siblings (market_state, submit_proof_request, etc.) by focusing on simulation of coordination regimes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by noting that supply is identical across regimes and only coordination changes, implying the tool isolates coordination effects. However, it does not explicitly name alternative tools or state when not to use it, stopping short of full usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• submit_proof_requestA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries full burden. It discloses the async workflow (Matchmaker assigns when capacity is free) and the return type (request id). However, it does not mention that this is a mutating operation, potential failure modes, or cancellation behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the action, and every sentence adds value. There is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with fully documented parameters and no output schema, the description adequately explains the return value and high-level workflow. However, it omits side-effect details (e.g., persistence, irreversibility) and error conditions, leaving some gaps in behavioral coverage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage with clear descriptions for each parameter (system, sizeUnits, deadlineTicks). The description adds no extra information about parameter meaning, so it does not exceed the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('submits') with a clear resource ('Proof Request to the market'), and specifies the Seeker side. It distinguishes itself by noting it returns a request id for polling, which differentiates it from sibling tools like poll_proof.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context: a Seeker submits when acting as a Seeker, and the returned id is meant for polling. It does not explicitly exclude alternatives like simulate_coordination or compose_workflow, but the Seeker-side framing implies appropriate use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compose_workflowA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description reveals that the tool runs the entire workflow without human-in-the-loop and returns an attested trace, which is meaningful behavioral context. It does not cover failure behavior, blocking/async execution, permissions, or side effects, so transparency is partial.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core concept, and each sentence earns its place by stating purpose, execution mode, step kinds, and return value. No redundant or filler text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers high-level intent, step taxonomy, and return value, which is solid for a complex orchestration tool. However, it omits details about the meaning and usage of sub-fields (detail, system, sizeUnits), the format of the attested trace, and behavior on failure, leaving gaps given the nested schema and lack of output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only describes the steps array but leaves the enum values opaque; the description adds semantic meaning by mapping each step kind ('observe a condition, request_proof, act, settle'). It does not explain detail, system, or sizeUnits, but the primary parameter is enriched.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'compose' and names the resource 'multi-step workflow', then details that it defines an ordered sequence autonomously executed by the surface. It distinguishes itself from sibling tools by being a meta-orchestration tool: it composes steps (observe, request_proof, act, settle) rather than performing a single operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear contextual guidance: use this when you want an agentic path with an ordered sequence that runs without human involvement. It does not explicitly name alternatives or state when not to use it, but the description makes the autonomous orchestration use case apparent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.