Server Quality Checklist

Tool Scores

• search_datasetsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals the returned fields (dataset ids, name, description, keywords, status), the status-preference guidance, the large corpus size (over 1,600 datasets), and the TS-prefix naming convention for census tables. It does not mention pagination or limits, but this is a search tool with no side effects and an output schema already supplies the return structure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is organized into three concise paragraphs: purpose, returned data, and search guidance. Every sentence contributes practical information, and the main purpose is front-loaded. It is slightly longer than strictly necessary but remains focused and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter search tool with an output schema, the description is remarkably complete. It tells the agent exactly what to search, how to phrase the search, which results to prefer, what the return will contain, and how to proceed to the next tool. No essential call-time information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes the query parameter thoroughly with field-targeting prefix syntax and bare-term behavior at 100% coverage. The description goes beyond the schema by adding practical guidance: prefer narrow terms, use 'TS058' as an example, and understand that census summary tables use a TS prefix. This adds real value without being redundant.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific, unambiguous statement: 'Find NOMIS datasets by keyword.' It names the exact resource (NOMIS datasets), the action (find by keyword), and distinguishes itself from the sibling get_dataset_dimensions by saying 'Start here, then call get_dataset_dimensions.' This is a clear, non-of-a-duplicative purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells the agent to start with this tool before calling get_dataset_dimensions, and provides a strong selection rule: prefer 'Current' status datasets over 'Historical' unless the user requested historical. It also advises using narrow search terms. It does not explicitly contrast with search_codes or fetch_data, but the resource-type distinction makes the primary use case obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_dataset_dimensionsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It clearly explains that the tool returns dimension concepts and geography type vintages, and it adds practical behavioral context about filter validity and downstream usage. It does not explicitly state auth requirements or side effects, but for a read-only listing tool the described behavior is sufficiently transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized and front-loaded with the core purpose. Each sentence earns its place: the essential prerequisite warning, the variability context, and the geography_types explanation with concrete examples. It is moderately long but dense with useful guidance, not redundant filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema, so return values are already documented. The description covers why the tool matters, how it relates to fetch_data and search_codes, and what the geographic_types values mean. For a discovery tool with one parameter, this is complete and actionable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single parameter, dataset_id, which is already described in the schema with an example. The description reinforces the importance of the dataset but does not add significant new parameter-level meaning beyond the schema. Baseline of 3 is appropriate because the schema handles parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'List the dimensions a dataset accepts as filters, and its geography types.' It clearly distinguishes this from siblings like fetch_data and search_codes by describing the unique role of dimension discovery. It also gives concrete examples of what the output contains, leaving no ambiguity about the tool's purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: 'Essential before fetching.' It explains that dimensions vary per dataset and that unlisted filters will be rejected or silently ignored by fetch_data, making the prerequisite relationship clear. It also names the next step, passing the chosen geography type to search_codes, which routes the agent correctly among sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetch_data_to_fileA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and discloses three non-obvious behaviors that cannot be inferred from the schema: nothing is 'loaded into memory or context,' output is capped at '25,000 records' without an API key, and the tool returns a path for later analysis. These traits directly affect whether an agent should select this tool and what to expect from it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four short passages, each earning its place: primary behavior, when-to-use examples, the API-key cap caveat, and the return value. The most decision-relevant sentence is front-loaded first, and no sentence is redundant with the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 4-parameter tool with an output schema, the description covers the call-relevant facts: use case, cap condition, auth hint, and return value. Minor gaps remain — no statement about overwriting an existing filename, failure behavior, or whether the returned path is locally accessible to the agent versus only on the server — but nothing essential to invoking it correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline of 3 applies; the schema already documents dataset_id, filters, filename, and select. The description adds little per-parameter detail, though it reinforces the 'unrestricted query' semantics of filters and that output lands in the server's download directory.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Opens with a specific action — 'Stream an unrestricted query to a CSV file' — and immediately contrasts it with the reply-returning behavior of the sibling fetch_data. The later phrase 'rather than paging fetch_data hundreds of times' explicitly names the closest alternative, so an agent can tell the tools apart without opening either schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives concrete when-to-use conditions: 'results too large to read' with realistic examples (whole geography types, long time series, LSOA-level census tables). It names fetch_data as the alternative and even routes to check_auth when the row count 'looks suspiciously round.' The only minor omission is an explicit when-not-to-use for small results, but that is strongly implied by 'instead of into the reply.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetch_dataA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full behavioral disclosure burden. It excellently warns that NOMIS returns HTTP 200 with silently truncated bodies, instructs agents to read the 'truncated' flag, and warns against aggregating incomplete results. This is high-value, non-obvious behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but every section earns its place: purpose, return fields, a critical truncation warning, a concrete example, and a clear boundary with fetch_data_to_file. The most important safety and usage information is front-loaded before the example.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity, absent annotations, and detailed schema, the description covers purpose, return values, failure modes, usage constraints, alternatives, and a realistic example. An output schema exists, so return-value details are not strictly required, but the description still provides them where they matter.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3; the schema already documents filter syntax, geography special values, time options, defaults, and constraints. The description adds a worked example and reiterates that codes come from search_codes, but it does not materially extend the parameter semantics beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description immediately states 'Fetch observations from a dataset', giving a specific verb and resource. It further clarifies scope with a worked example and distinguishes the tool from the sibling fetch_data_to_file by describing when that alternative should be used.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage guidance is explicit: codes must come from search_codes, truncated results must not be aggregated, and fetch_data_to_file should be used for oversized or whole-geography-type queries. This clearly routes agents to the correct tool under the right conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_authA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does so thoroughly: it discloses the silent 25,000-record cap, the misleading HTTP 200 behavior, the probe's narrow-column download with a few-second runtime, and the key's redaction from URLs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than average, but every sentence earns its place: purpose, silent cap warning, run conditions, setup steps, and security considerations are all operationally relevant and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter diagnostic tool with an output schema, this is complete: it states what is checked, why it matters, when to invoke it, how long it takes, and how to configure the prerequisite. Nothing an agent needs to call it correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema already exhaustively covers inputs. The description adds useful context by clarifying that the key is supplied via NOMIS_UID or NOMIS_CONFIG_FILE, not as a tool parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and object: 'Report whether the server holds a NOMIS API key, and test the cell limit.' This clearly distinguishes check_auth from sibling data-access tools such as fetch_data and search_datasets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to run: 'Run this when a fetch reports truncation, or to confirm setup.' It also explains the consequence of a missing key and how to supply one, giving an agent unambiguous context for invocation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_codesA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses key behaviors: codes are opaque integers that cannot be guessed, geography search without type_code returns the type list, and a geography search at the top-level codelist returns success with no matches. This is valuable operational transparency beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense and every sentence earns its place. The core purpose is front-loaded, followed by essential behavioral caveats and concrete examples. It is longer than average, but the complexity of the two-step geography behavior justifies the length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity, an output schema exists, and there is no annotation coverage, the description is complete. It covers the main use case, the tricky geography case, the fallback behavior, and example calls. An agent has enough context to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 100%, the description adds significant semantic value: it explains the relationship between concept='geography' and type_code, shows concrete call examples, and clarifies that type_code is mandatory for geography but unnecessary for other dimensions. This goes well beyond the schema's property descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description immediately states the tool's purpose: 'Resolve dimension values to the numeric codes fetch_data needs.' This is a specific verb plus resource and clearly distinguishes it from siblings like search_datasets and get_dataset_dimensions by explaining it produces the codes required by fetch_data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: 'Always come here first.' It explains the two-step geography workflow, when type_code is needed versus omitted, and that non-geography dimensions are searched directly. This leaves little ambiguity about how to use the tool correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.