Server Quality Checklist

Tool Scores

• trust_profileA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden and does disclose key edge cases: sections are null when a source has no data and found=false when the trust isn't in the identity mapping. It does not mention any side effects, auth, or rate limits, but for a read/composite tool these may be less critical; output schema can cover return details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, no filler: purpose first, then identity parameters, then fallback/null behavior. Every sentence adds a distinct piece of knowledge an agent needs.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The core behavior, identity resolution, nullable sections, and not-found case are all covered, and an output schema exists for return shapes. The main remaining gap is valid specialty values, but the description is otherwise sufficient for correct tool selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It explains identifier as an RTT provider code or trust name and defines by_name=true for names, but specialty is only implied by context with no accepted values, format, or examples, leaving one required parameter underspecified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific composite verb+resource: builds a combined profile for one NHS trust and specialty, joining three named sources. This clearly differentiates it from siblings like wait_time_trend and get_trust_rating, which presumably cover a single source.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description establishes clear context for when to use it — when current wait, RTT trend, and CQC rating are needed together. It also gives conditional guidance for identifier mode via by_name, but doesn't explicitly name sibling alternatives for single-source queries, so no exclusions are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rank_trusts_by_waitA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry behavioral disclosure, and it does: it reveals the ordering, the optional filters, and the important edge-case behavior that trusts without figures are listed after ranked ones. It does not mention read-only status or availability constraints, but for a data-query tool the stated behavior is substantively transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences contain the core purpose, data source, ordering, optional parameters, and an edge-case behavior with zero filler. The statement is front-loaded with the action and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a ranking tool with an output schema, the description is largely complete: it explains the data source, ordering, filters, and gap behavior. Minor details like the unit of waiting time or whether limit applies to gap-listed trusts are not stated, but an agent can invoke the tool correctly from this description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% parameter coverage, so the description must compensate, and it maps all three parameters: specialty is the required filter, region restricts geography, and limit caps the number of returned trusts. It does not enumerate valid specialty/region values, but it conveys the role of each parameter beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific action ('Rank NHS trusts by their latest waiting time for a specialty'), specifies ordering ('longest first'), and identifies the data source. This clearly differentiates it from siblings like wait_time_trend or lookup_wait_time, which are point/trend lookups rather than cross-trust rankings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states the use case clearly—comparing trusts across a specialty and optionally filtering by region or count. It does not explicitly name sibling alternatives or say when not to use it, but the ranking purpose and data source give enough context for an agent to select it over lookup/trend tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_trust_ratingA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It discloses the return shape (overall plus per-domain ratings), the not-found behavior (found=false), and an important instruction not to infer a rating. This is strong for a read-only lookup tool, though it does not explicitly mention safety or data freshness.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place: purpose and input, return content, and the critical not-found behavior. Information is front-loaded and no filler is present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter lookup tool with an output schema, the description is complete. It covers input, expected outputs, the not-found case, and the correct behavior when no rating exists. Nothing essential for an agent to select and call the tool correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It explains that cqc_provider_id is the CQC provider id and supplies a realistic example ('1-101681210'), adding meaning beyond the bare schema field. It does not elaborate on id source or formatting rules, but the example and context are sufficient for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action (look up), the resource (a trust's current CQC quality rating), and the lookup key (CQC provider id), with a concrete example. It also distinguishes itself from sibling tools focused on wait times and trust profiles by explicitly focusing on CQC quality ratings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states the task and input clearly enough that an agent knows when to invoke it: whenever a trust's CQC rating is needed. It does not explicitly name sibling alternatives or exclusion conditions, but the focused wording and the contrast with wait-time-related siblings imply the appropriate use case.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• wait_time_trendA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral disclosure burden and does an excellent job. It discloses the output return fields (start/end values, change, percentage, direction, full monthly series), the units (weeks), and the found=false edge case, explicitly instructing not to invent a trend when data is absent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact yet information-dense: purpose, units, input guidance, output summary, and edge-case handling are all included in three sentences. Every sentence earns its place and no filler exists.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's modest complexity, an output schema, and no annotations, the description is complete. It covers the data source, input semantics, units, expected return values, and the missing-data behavior, so an agent has everything needed to invoke it correctly and interpret results appropriately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for both parameters. It does by explaining that provider_code is an NHS trust code with an example ('RGT') and specialty is a clinical specialty with an example ('Cardiology'), giving an agent enough semantic grounding to call the tool correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Summarise') and a specific resource ('waiting time for a given NHS trust and specialty... moved over time'), which distinguishes it from sibling tools like lookup_wait_time or rank_trusts_by_wait. It also previews concrete outputs, leaving no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies when to call this tool: when a trend summary over time is needed, rather than a point lookup or ranking. It gives concrete input requirements (provider code, specialty) and an explicit no-data behavior, but it does not explicitly name alternative tools or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_wait_timeA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states that waits are in whole weeks, data is updated weekly, found=false is returned when the trust/specialty is not held, and that the agent must not invent a figure. These are material behaviors beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Every sentence adds necessary information: purpose and source, units, parameter formatting, not-found behavior, and the sibling-tool distinction. The description is front-loaded with the core purpose and contains no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter lookup tool with an output schema, this description is remarkably complete. It covers source, freshness, units, input formatting, missing-data behavior, and the key alternative tool, leaving no significant gap for an agent to resolve.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does so thoroughly by explaining that provider should be 'the trust name as published' with an example, and specialty should be like 'Cardiology' — also clarifying the name-based identifier, which is essential for correct use.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Look up the latest reported waiting time for one NHS trust and specialty from My Planned Care.' It clearly distinguishes this tool from wait_time_trend by noting the trust is identified by name here rather than by RTT provider code.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly contrasts this tool with wait_time_trend, stating that lookup_wait_time uses the trust name while wait_time_trend uses an RTT provider code. This gives the agent a concrete basis for choosing between the two alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.