Server Quality Checklist

Tool Scores

• datasets_listA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of disclosing behavior. It explicitly states the return contents and the filtering behavior (matching title or description), which is useful. However, it omits details like pagination, ordering, or case sensitivity, leaving some behavioral aspects untold.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, starts with the core purpose, then lists return fields, then describes the optional filter. Every sentence carries useful information with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite lacking an output schema, the description explicitly names the return fields, which is sufficient for a simple listing tool. With only one optional parameter and a straightforward purpose, the description covers the essential context for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides 100% coverage for the 'search_query' parameter, including the note that it filters by title or description. The description adds examples ('prescribing', 'dental') but otherwise repeats the schema. Baseline for high coverage is 3, and the examples provide marginal added value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists all available datasets on the NHSBSA Open Data Portal and specifies what it returns (names, descriptions, resource counts). This is a specific verb+resource+scope. However, it does not explicitly differentiate from sibling tools like datasets_metadata or datasets_query, though the listing purpose is distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for listing datasets with optional filtering by search term, but it does not provide explicit guidance on when to choose this tool over siblings such as datasets_metadata or datasets_query. There are no exclusions or alternative recommendations, making the usage context adequate but not fully explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• datasets_queryA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It discloses filtering and pagination behavior, but omits details like return format, error handling, authentication needs, or side effects. This is adequate but leaves gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, using three short sentences to convey purpose, prerequisite, and capabilities. Every sentence earns its place, with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema, and the description does not mention what the response looks like or how to handle errors/empty results. It covers the core usage but leaves out some context that would be helpful for a generic query tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes all four parameters with 100% coverage, so the description adds limited value beyond reaffirming the resource_id source and pagination. The mention of 'filtering by field values' aligns with the filters parameter but does not introduce new semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Query a specific dataset resource') and the target resource, distinguishing it from sibling tools like datasets_list and datasets_metadata. It also mentions the required resource ID, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage context by instructing to use datasets_metadata first to obtain resource IDs, and notes the tool supports filtering and pagination. It does not explicitly state when not to use this tool versus alternatives, but the prerequisite guidance is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• datasets_metadataA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool returns metadata and resource IDs, and that actual data is not included (since it directs to datasets_query). However, it does not mention return structure, error behavior, or any permissions, so it remains moderately transparent for a simple read-only metadata tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main purpose and followed by a clear usage instruction. Every sentence earns its place with no redundancy, achieving excellent conciseness and structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description covers the essential context: what the tool returns (metadata, resource IDs) and how to proceed (use datasets_query). It lacks a detailed list of returned fields, but this is not critical for a straightforward metadata fetch. The sibling tool context further clarifies its role.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, providing an example for dataset_id. The description adds context by explaining the purpose of the ID (to retrieve metadata for a specific NHS dataset and obtain resource IDs), but it does not add technical details beyond the schema. The baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb and resource: 'Get detailed metadata for a specific NHS dataset'. It also distinguishes itself from siblings by mentioning that it returns resources (data files) and their IDs, which is not a function of datasets_list (listing) or datasets_query (querying data).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs when to use the returned resource ID with the datasets_query tool, giving clear cross-tool guidance. It implies this tool is for retrieving metadata before querying data, though it does not explicitly state when not to use it or compare with datasets_list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• prescriptions_cost_analysisA
  4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing behavioral traits. It mentions the scope of data but does not explicitly state that the operation is read-only, lacks any side effects, or requires specific permissions. It also does not describe response structure, pagination, or error behaviors. Given the lack of annotations, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the primary action, and includes only essential information. The first sentence states the purpose, and the second adds scope and an explicit alternative. There is no redundant or extraneous content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema, so the description should offer some idea of what the tool returns. It does by saying 'Shows aggregate costs and volumes.' For a tool with five parameters and a relatively simple query purpose, the description is adequately complete, though it omits details like pagination or resource auto-discovery (which are in the schema).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description does not add details about any specific parameter beyond what is already in the schema, such as the meaning of limit or bnf_code. It only adds context about the national-level aggregation, which is not parameter-specific. The schema already provides clear descriptions for all five parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get prescription cost statistics from the national Prescription Cost Analysis (PCA) dataset.' It explicitly notes the national-level scope and differentiates from the sibling tool by directing users to prescriptions_search for practice-level data, satisfying the requirement for a specific verb, resource, and sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage context: it is for national aggregate costs and volumes, and it explicitly says 'not by practice.' It also names the alternative tool for practice-level data ('Use prescriptions_search for practice-level data instead'), giving clear when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• prescriptions_searchA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It states what the tool returns and imposes a parameter requirement, but it does not disclose default dataset behavior, pagination, error handling, or explicitly confirm it's read-only. Some behavioral traits are implied (search implies read) but not fully specified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no redundant information. It front-loads the purpose and immediately adds the key usage constraint and return details. Every sentence contributes value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, the description adequately indicates the return fields (items dispensed, quantities, costs). It also mentions the parameter requirement and available filters. However, it does not describe the list/response structure in more detail or note the default dataset and paging behavior, though some of that is in the schema. It is sufficient for a straightforward search tool but not exhaustive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds a high-level grouping of search parameters ('BNF code, drug name, practice code, or time period') but does not add significant depth beyond the schema's per-parameter descriptions. The constraint on providing at least one parameter is a useful addition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Search' and the specific resource 'NHS prescribing data from the English Prescribing Dataset (EPD)'. It enumerates the filters (BNF code, drug name, practice code, time period) and return fields (items dispensed, quantities, costs), which distinguishes it from sibling tools like spending trends or cost analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool: to find prescription records. It also gives an explicit usage constraint ('At least one search parameter should be provided'). However, it does not explicitly mention alternatives or when not to use this tool, leaving some ambiguity relative to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• organisations_relationshipsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It explicitly states what is returned ('relationship types, target organisations, and status'), which is transparent about the data output. Although it does not mention authentication or rate limits, the read-only nature is implied by 'Get' and the focus on return content suffices for a simple read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the main purpose, followed by examples and return structure. Every sentence contributes meaning, with no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple one-parameter tool with no output schema, the description sufficiently explains the purpose, provides examples, and outlines the return fields. It could slightly expand on the full set of relationship types available, but it is adequately complete for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage with a clear description of the ods_code parameter including examples. The tool description adds value by explaining the context of the parameter within relationships but does not add additional semantic detail beyond the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets relationships for a specific NHS organisation, with concrete examples (ICB commissions GP practice, region for trust). This clearly distinguishes it from sibling tools like organisations_get or organisations_search, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: for a specific NHS organisation when you need its relationships. It does not explicitly compare with alternatives or state when not to use it, but the examples and the 'specific NHS organisation' wording make the use case evident.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• prescriptions_spending_trendsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the data source (OpenPrescribing), the time range (last 5 years), the granularity (monthly), and the returned fields (items, quantities, costs). It also mentions the filtering option by organisation type. This is more than basic transparency, though it does not discuss pagination or error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences long, each contributing essential information (purpose, data source, return values, filtering). It is front-loaded with the main action and avoids unnecessary detail. No filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 3 parameters (one required) and no output schema, so the description must explain what the tool returns. It covers the monthly items, quantities, costs, and the organisation filter. However, it does not explicitly state that the output is a time series or how the data is aggregated by org type, but this is partially implied. Overall, it is sufficiently complete for an analysis tool, with a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already explains all parameters thoroughly (bnf_code, org, org_type) with examples. The description adds no new parameter-level meaning beyond restating 'drug or BNF category' and 'filtered by organisation.' Hence, it does not exceed the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Analyse prescribing spending trends over time for a drug or BNF category.' The verb 'analyse' plus the resource 'prescribing spending trends' and the temporal scope ('over time') is specific. It distinguishes itself from siblings like prescriptions_search or cost_analysis by emphasizing the time trend aspect.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: this tool is for analysing trends over time, using data from OpenPrescribing, filtering by organisation. However, it does not explicitly mention when not to use it or point to alternative tools for different use cases (e.g., cost comparison). The 'over time' phrasing implies the intended use, but exclusions are absent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• organisations_searchA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the full burden of disclosing behavior. It states the data source (ODS ORD directory), the return payload (ODS codes, names, statuses, primary roles), and the requirement for at least one search parameter. It doesn't mention pagination details or error handling, but the schema covers limit/offset/status defaults.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: two sentences that front-load the core purpose, then add the requirement and return info. Every clause earns its place; no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 6 parameters, no output schema, and no annotations, the description covers the essential high-level semantics: what it searches, what it returns, and the input constraint. The schema fills in parameter defaults and enums. It could be more complete by mentioning pagination behavior or explicitly directing to organisations_get when an ODS code is known, but the current description is adequate for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds value by explicitly naming the three searchable parameters (name, role, postcode) and clarifying that at least one is required, which is not obvious from the schema since all properties are optional. It also highlights that combinations are allowed ('at least one').

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear verb+resource: 'Search for NHS organisations', then specifies the search axes (name, role, postcode) and the directory (ODS ORD). It also lists returned fields, distinguishing it from sibling tools like organisations_get (likely by ODS code) and organisations_relationships.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: search by name/role/postcode, and mandates 'At least one search parameter (name, role, or postcode) is required.' It implies the tool is for finding organisations when you don't have an ODS code, though it does not explicitly contrast with alternatives like organisations_get.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• organisations_getA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the full burden of behavioral disclosure. It clearly states the tool is a read operation ('Get') and enumerates the returned fields, which gives a good sense of what the agent can expect. However, it does not mention error behavior, authentication requirements, or rate limits, and it doesn't explicitly label itself as read-only. A slightly more detailed description (e.g., noting that it returns the full organization record, or that it is safe to call) would push this to 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences and highly efficient. The first sentence delivers the core purpose and expected return data, and the second gives actionable guidance. No filler or redundant information. It is front-loaded and easily scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple get-by-ID tool with a well-defined parameter and no output schema, the description is complete enough. It covers what the tool does, what it returns (the key data fields), and how to obtain the required parameter. The presence of sibling tools like organisations_search further contextualizes this tool within the broader workflow. No significant gaps remain for the agent to be left uncertain about when or how to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage for the single parameter ods_code, so the baseline is 3. The description adds value by framing the ODS code as the key identifier and instructing the agent to obtain it via organisations_search first. This goes beyond the schema's simple definition and provides contextual meaning for the parameter's usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get full details for a specific NHS organisation by its ODS code.' It includes a specific verb (Get) and resource (organisation details), and lists the exact fields returned (name, address, roles, relationships, succession history). This distinguishes it from siblings like organisations_search (which finds ODS codes) and organisations_relationships (which may only return relationship data).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides usage guidance: 'Use organisations_search first to find the ODS code.' This tells the agent when to use this tool versus the search tool, and sets a clear prerequisite for invocation. It effectively communicates the typical workflow context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.