Server Quality Checklist

Tool Scores

• okf.prepare_mcp_planB
  3.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint. The description adds behavioral context by stating it preserves unresolved dimensions, rejects credential fields, and never authorizes or executes, which goes beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief (two sentences) but the first sentence is dense with jargon, making it less clear. It lacks a clear breakdown of what the tool does in simpler terms.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (5 parameters, no output schema, nested objects), the description lacks details on the return value or how the plan is used afterward. It does not cover what the output contains, leaving the agent with insufficient context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description refers to 'audience, purpose and caller-declared expiry,' linking to three parameters, but does not explain 'arguments,' 'record_id,' or their semantics. With 0% schema coverage, the description should provide more parameter detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it binds a frozen identity and snapshot digest to tools, audience, purpose, and expiry, and emphasizes it never executes. The title 'Prepare a non-executing MCP selection plan' reinforces this. However, jargon like 'frozen identity' and 'snapshot digest' may not be universally understood, slightly reducing clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies it should be used for preparing a plan without execution ('Never authorises or executes a query') and mentions preserving unresolved dimensions and rejecting credentials. It does not explicitly contrast with siblings like okf.get_record or okf_eval.submit_answer, but the distinct roles are inferable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• okf_eval.submit_answerA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds context beyond annotations: it specifies the tool is non-persisted, deterministic, and never accepts hidden reasoning. Annotations already mark it as readOnlyHint=true, idempotentHint=true, destructiveHint=false, so the description reinforces and elaborates on these traits, providing useful behavioral clarity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that effectively front-loads the core purpose (validate, deterministic, non-persisted) and lists accepted inputs. Every phrase adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (11 parameters, 8 required, no output schema, nested objects), the description is insufficient. It does not explain the structure of the return envelope, the meaning of required parameters, or how inputs like task_id, chosen_record_id, and considered_record_ids should be used. The absence of output schema makes this gap more critical.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description carries full responsibility for explaining parameters, but it only mentions 'visible answers, evidence and caveats,' corresponding to answer, evidence, caveat_ids. Eleven parameters exist, including key ones like task_id, chosen_record_id, considered_record_ids, alternatives, mcp_plan, contrasts, substitution, confidence, all unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Validate and return a deterministic, non-persisted evaluation envelope.' It specifies what it accepts (visible answers, evidence, caveats) and excludes (hidden reasoning), distinguishing it from siblings like okf.get_record, okf.compare, and okf.prepare_mcp_plan, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when or when not to use this tool versus alternatives. No explicit context for usage, prerequisites, or exclusions are mentioned, leaving the agent to infer when submit_answer is appropriate given the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• okf.compareA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true. The description adds the behavioral caveat 'Never asserts statistical equivalence,' which goes beyond annotations. However, it does not disclose error handling or behavior with missing records.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with the primary action front-loaded. Every sentence adds value, with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool and rich annotations, the description is adequate but could be more complete by explaining the output format or providing an example. Parameter explanation is minimal.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate. It adds the context of 'two to five exact records' reinforcing minItems/maxItems constraints but does not explain the format or meaning of the record_ids strings beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool compares exact ONS metadata records by returning field contrasts. It specifies the range (2-5 records) and differentiates from siblings like okf.get_record which retrieves a single record.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for comparing two to five exact records but does not explicitly state when to use this tool versus alternatives like okf.get_record. It provides no when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• okf.get_recordA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint and idempotentHint. Description adds 'frozen' to indicate immutability but does not disclose error behavior (e.g., what happens if identifier not found). Minimal additional transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no fluff, front-loaded with action and resource. All words carry meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema; description does not explain return structure beyond 'complete frozen metadata record'. Lacks details on pagination, error responses, or data format. Adequate but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has no description for 'identifier' (0% coverage). Description compensates by listing acceptable identifier types: 'exact stable ID, native ID, route or declared evaluation alias', which adds meaningful context beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses specific verb 'Resolve' and lists exact resource types (stable ID, native ID, route, alias) and output ('complete frozen metadata record'). Clearly distinguishes from sibling tools like okf.compare or okf_eval.submit_answer.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. While the purpose implies single-record retrieval, it does not state when not to use it or mention alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.