Server Quality Checklist

Tool Scores

• search_apisB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must cover behavioral traits. It mentions that it searches for live endpoints serving Norwegian public data, but omits details on rate limits, authentication, error handling, or what happens on empty results. The description lacks sufficient transparency for an unannotated tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two efficient sentences: the first states the purpose and provides examples, the second gives a clear follow-up action. No redundant information; front-loaded and concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description should indicate the response structure (e.g., presence of id, total results). It only mentions IDs for endpoint URLs, leaving the agent uninformed about other fields or pagination details. The description is incomplete for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with each parameter having a description. The description adds domain context (Norwegian public data) and notes that returned IDs can be used with get_api, but does not add new meaning to the parameters themselves. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's verb ('Search') and resource ('APIs / data services' on data.norge.no), with specific examples (Statistics Norway, Kartverket, Brønnøysundregistrene) that distinguish it from siblings like search_datasets (which searches datasets).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a follow-up action ('Use get_api with a returned id for endpoint URLs') but does not explicitly state when to use this tool over alternatives like search_datasets or fetch_data. Usage context is implied but not fully elaborated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_datasetsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must convey behavioral traits. It mentions return fields and language preference but does not discuss pagination, rate limits, or error handling. The schema covers pagination details, so the description adds limited behavioral context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the main purpose, and every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 5-parameter search tool with no output schema and no annotations, the description provides a basic overview but lacks details on pagination, filtering, and error scenarios. It is adequate but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds value by noting language preference and linking to get_dataset, but it does not explain parameter usage beyond what the schema already describes.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches for datasets on data.norge.no by keyword or topic, specifies return fields (id, publisher, themes, short description), and differentiates from sibling tools like get_dataset (which gets download links) and search_apis (likely different resource).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises that Norwegian search terms match best and suggests using the returned id with get_dataset for download links, giving practical usage guidance. However, it does not explicitly state when not to use this tool versus alternatives like search_apis.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_datasetA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description covers the return content (metadata, distributions, URLs, format, licence) and implies a read operation. It doesn't address authentication, rate limits, or side effects, which is acceptable for a simple fetch tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the action, no unnecessary words, and efficient communication of purpose and value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple tool (1 param, no output schema), the description sufficiently explains what is returned and how it fits into the workflow, making it complete for an AI agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%; the param 'id' already has a clear description in schema. Description reinforces its origin from search_datasets but adds no further semantic depth beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Fetch full metadata for one dataset by its id' with specific details about contents (distributions, URLs, format, licence) and explains that this is how to find where to actually get the data, distinguishing from siblings like search_datasets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description implies using search_datasets first to get the id, and positions this as a step before fetching actual data. It doesn't explicitly name alternatives, but context from sibling tools provides differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_apiA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Although no annotations are provided, the description discloses the read-only nature of the operation and specifies what data is returned (endpoint URLs, machine-readable description link). It does not mention potential side effects, which are minimal for a fetch operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that immediately conveys the tool's purpose and key details. Every word adds value, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description hints at important return fields (endpoints, machine-readable link) and is sufficient for a simple retrieval tool. However, it could be more explicit about the full set of metadata fields returned.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of parameters with a clear description for the 'id' parameter. The tool description reiterates the parameter's purpose but does not add new meaning beyond the schema, aligning with the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches full metadata for one API/data service by its id, and specifies key included elements (endpoint URLs, link to OpenAPI spec). It distinguishes itself from sibling tools like search_apis which searches for APIs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description emphasizes using the id from search_apis, implying a prerequisite and a workflow. It provides clear context for when to invoke the tool, though it lacks explicit when-not-to-use guidance or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetch_dataA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses return behavior: text formats return content, large responses are truncated, binary returns metadata only. This goes beyond a simple 'fetch' description. However, it omits authentication needs, rate limits, or error handling specifics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, no redundant information. The description is front-loaded with the key action and resource, then adds return behavior and usage guidance. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 params, no output schema), the description covers main scenarios: what URLs to use, what is returned for text vs binary, and size limits. It does not cover edge cases like invalid URLs or authentication, but for a straightforward fetch tool it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and both parameters have descriptions in the schema. The description adds context about maxKilobytes (truncation behavior) and mentions text formats, which relates to the url parameter's effect. This provides slight additional meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool downloads data from specific URL types (downloadURL, accessURL, API endpoint, etc.) and contrasts it with sibling tools by noting it retrieves the data itself, not catalogue metadata. The verb 'download' with resource 'data' is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use this tool (after obtaining a data URL from get_dataset or get_api) and implicitly when not to (for metadata). It names alternatives (get_dataset, get_api) but does not explicitly state exclusions like 'do not use for catalogue lookups.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.