Server Quality Checklist

Tool Scores

• search_costsA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It only states it returns real prices and requires no key, but lacks information on data freshness, rate limits, permissions, or whether it is read-only. Minimal behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is extremely concise with two sentences that front-load the core purpose and key usage hint. No redundant or verbose text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the description covers the purpose and one optional parameter, it does not explain the output format (e.g., list of agents with prices) or any limits, which is important given no output schema. Adequate but missing output details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with the parameter description already present. The description adds no new semantic meaning beyond the schema for 'max_price_pence'. Baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool retrieves real price_per_call_pence for every agent, distinguishing it from sibling tools like search_agents (general search) and search_reliability (reliability data). Verb 'search' combined with 'pricing' is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates use case: budget-aware agent selection before invoking. It mentions 'No key needed' but does not explicitly state when not to use or provide alternatives, though sibling tool names imply other tools for different aspects.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_reliabilityA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry full behavioral transparency. It describes the tool as returning real system measurements without side effects, and mentions 'No key needed' implying no authentication. However, it lacks details on rate limits, permissions, or whether it is idempotent, which would be expected for a read-only tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the core purpose, and contains no unnecessary words. Every sentence adds value, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one optional parameter, no output schema), the description covers the key aspects: what data is returned, how it relates to siblings, and that no key is needed. However, it could briefly mention the expected output format or that it returns an array of objects, since no output schema exists.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single optional parameter 'agent_slug', so the schema already documents it well. The description does not add new meaning beyond stating 'No key needed' which is about the tool, not the parameter. Thus, baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns 'Real, system-measured reliability per agent' with specific fields (success_rate, avg_latency_ms, sample_size). It distinguishes itself from sibling 'search_agents' by noting it exposes the same health data standalone for reliability-focused queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates when to use it ('reliability-focused queries') and references the sibling 'search_agents' as containing the same data. However, it does not explicitly exclude other scenarios or mention when not to use it, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• invoke_agentA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses cost (spends balance), authentication requirement (FD_API_KEY), return values (output, charge, proof_id), error handling (honest declines, insufficient balance), and idempotency (never double-charges). Missing explicit timeout behavior, but schema covers that.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with core purpose, followed by key behavioral details. No wasted words; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and moderate complexity, description covers purpose, prerequisites, return values, error handling, and idempotency. Lacks exact output format and handling of unexpected errors, but sufficient for basic usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline 3. Description adds context that agent_slug can be discovered via search_agents and mentions return values for outputs, but doesn't significantly enhance parameter understanding beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it invokes a ForceDream agent to do real work, mentioning spending balance, API key requirement, and returns output/charge/proof_id. It distinguishes from siblings search_agents and verify_proof by referencing them as discovery and verification tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides context for when to use the tool (invoke agent for work) and mentions alternatives (search_agents for discovery, verify_proof for verification). States it invokes once and never double-charges, but doesn't explicitly exclude scenarios where other tools would be better.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_agentsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that metrics are 'honest, system-derived' and every agent has 'real cryptographic proofs'. No annotations provided so description carries full burden; sufficiently transparent for a search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, first sentence immediately states purpose. No redundant information. Efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, but description mentions key return fields (proof_count, success_rate) and verifiability. Could note pagination or sorting, but adequate for a simple search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already describes both parameters (100% coverage). Description adds clarification on query match fields, but doesn't significantly enhance semantic understanding beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb 'discover' and resource 'ForceDream agents' with specific metrics (proof_count, success_rate). Distinct from siblings invoke_agent and verify_proof.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states 'No key needed' and optional filters. Links to verify_proof for verification, implying when to use that sibling. Could be more explicit about when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_providersA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses real-time nature, lists output fields, mentions internal use by adaptive routing, and states no authentication required. Lacks explicit read-only statement but fairly transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. Front-loaded with the core purpose and immediately lists relevant attributes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless tool with no output schema, the description provides sufficient context: what data is returned and that no key is needed. Could mention return format but adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters, so baseline 4. The description adds value by explaining what the tool returns, though no parameter details needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns 'Real, live inference-provider health' with specific attributes (health_score, breaker_state, uptime_ratio, recent successes/failures). This distinguishes it from siblings like search_costs or search_reliability.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for real-time health checking and notes 'No key needed', but does not explicitly say when to use it vs alternatives like search_reliability or search_costs. No when-not advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_proofA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Fully discloses that verification is local, uses Ed25519 math, and does not contact ForceDream. No annotations exist, so the description carries the burden and meets it well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences, front-loaded with purpose. No redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, inputs, and verification mechanism. Lacks mention of output format (e.g., returns boolean or proof details), but absence of output schema reduces expectation. Still adequate for a simple verification tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% coverage, and description adds meaning by explaining the fetch vs direct verify distinction and optionality. Adds value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the verb 'verify' and the resource 'ForceDream agent proof', specifying the use of public-key cryptography. Purpose is distinct from sibling tools 'invoke_agent' and 'search_agents'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Describes two ways to provide input (task_id or proof object) and clarifies local execution with no account needed. Implicitly guides usage but lacks explicit when-not-to-use or alternatives beyond context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.