Server Quality Checklist

Tool Scores

• get_facetsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It only says 'get' and 'discovering', which implies a read operation, but does not explicitly state safety, side effects, authentication needs, or return format. This is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences. The first sentence states the core function, and the second adds practical context. There is no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, the description should clarify what the tool returns and how to use the optional parameter. It lacks details on return structure, default behavior when 'facet' is null, and any limitations. The description is minimal and leaves the agent guessing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one parameter 'facet' with no description, and the schema description coverage is 0%. The tool description does not explain how to use the 'facet' parameter, what valid values are, or whether it is required. It only mentions types of facets without linking them to the parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'gets available filter values for categorizing datasets' and lists examples like publishers, keywords, and themes. This specific verb+resource pairing distinguishes it from siblings that deal with datasets/records rather than facets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Useful for discovering publishers, keywords, themes, or other facets to refine dataset searches' gives clear context for when to use this tool. It does not explicitly name alternatives or exclusions, but the use case is well implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_recordsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It reveals important behavioral traits: ODSQL syntax, optional filters, ordering, and pagination. However, it does not explain return format, default pagination behavior, or error handling, leaving gaps for a query tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose. Every word adds value, and the structure is efficient for a tool description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 12 parameters, no annotations, and no output schema, the description is notably incomplete. It gives only a surface-level overview and does not clarify the ODSQL syntax or the semantics of the many optional parameters. This is insufficient for an agent to use the tool correctly without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions high-level concepts like WHERE, ordering, and pagination, but fails to map them to actual parameters (where, order_by, limit, offset). Many parameters (select, refine, group_by, timezone, include_links) remain completely unexplained, making proper invocation difficult.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Query and filter records from a dataset using ODSQL syntax.' It specifies the verb (query/filter), resource (records from a dataset), and distinguishes itself from siblings by emphasizing 'actual data' versus metadata or export functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: 'Use this to retrieve actual data from a dataset with optional WHERE clauses, ordering, and pagination.' This tells the agent when to use the tool, though it does not explicitly mention exclusions or alternative tools. It lacks a direct comparison to siblings like get_dataset or get_facets.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• export_dataset_urlA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden. It explains that the tool generates a URL, implying no direct data manipulation, but does not explicitly state side effects, expiration, or permission requirements. Thus it provides minimal but non-contradictory behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action, and contains no filler. It is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's purpose and usage context, and an output schema exists to define return values. However, the 'where' parameter is left unexplained, which is a notable gap for a tool with three parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must compensate. It adds format examples for the 'format' parameter, but does not explain 'where' or 'dataset_id' beyond their schema titles. This is insufficient for a low-coverage schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the tool's function: generating a download URL for dataset exports. It lists supported formats and distinguishes itself from sibling tools that fetch datasets/records/facets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides an explicit use case: 'Use this when you need to download or share dataset exports.' However, it does not explicitly mention alternatives or when not to use this tool, so it scores a 4 rather than a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_datasetA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the full burden of behavioral disclosure. It explains what the tool returns (metadata fields), which is useful, but it does not explicitly state that it is read-only, nor does it mention error conditions, auth needs, or any caveats. The 'get' verb hints at safety, but it is not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action, and every sentence adds value. It efficiently combines purpose, output contents, and usage context without any filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a single simple parameter and no output schema, the description is fairly complete: it states what is returned (field definitions, schema, publisher info, record count) and when to use it. It does not detail error responses or output format, but for a metadata retrieval tool, this is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides only a required dataset_id of type string with 0% description coverage. The description adds minimal context by saying 'for a specific dataset,' but it does not explain how the ID maps to the resource, where to find it, or its expected format. The description does not compensate sufficiently for the lack of schema-level parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves detailed metadata for a specific dataset, listing the types of metadata (field definitions, schema, publisher info, record count). This stands out from siblings like get_datasets (likely a listing) and get_records (querying records), making the purpose immediate and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use: 'Use this to understand a dataset's structure before querying records.' This implies a workflow and differentiates from get_records, but it does not explicitly name alternatives or state when not to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_datasetsA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses key behavioral traits: it only searches open datasets, and it has two modes with distinct matching behavior. Semantic mode handles synonyms and other languages, lexical does exact matching. Since there are no annotations, this is helpful context, but it omits details such as pagination behavior or rate limits, which are often relevant for search tools.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long and front-loads the purpose. It succinctly explains the two modes and provides usage guidance without extraneous words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has nine parameters and no annotations or output schema, so the description needs to be quite thorough. It covers the core purpose and search modes, but it does not explain the majority of the parameters or the structure of the returned list. This leaves significant gaps for an agent trying to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has nine parameters with zero descriptions, so the description must compensate. It explains 'search_mode' (semantic vs lexical) and implies the 'search' parameter, but leaves limit, offset, refine, exclude, order_by, timezone, and include_app_metas entirely unexplained. Given the low schema coverage, this is insufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Search and list available open datasets') and identifies the source ('data.bs.ch'). It distinguishes itself from sibling tools by being the plural search tool, while 'get_dataset' implies retrieving a specific dataset. The two search modes further clarify its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use each search mode: 'Use semantic for conceptual discovery, lexical for precise term/name lookups.' This is clear context, but it does not explicitly mention when to prefer sibling tools like 'get_dataset' for fetching a specific dataset, though the purpose implies the distinction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.