Server Quality Checklist

Tool Scores

• track_updatesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the output format (compact list with name, administrator, change time, change type, link). However, it does not mention any behavioral traits like rate limits, authentication needs, or side effects, which is acceptable for a read-like monitoring tool but still a gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise with two sentences. The first sentence states the purpose, and the second describes the output. No extraneous information, front-loaded effectively.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given three parameters and no output schema, the description covers purpose, filter options, and return fields. It omits details like ordering or default limit behavior, but is largely complete for a simple monitoring tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 67% (topic and organization have descriptions; limit has default/min/max but no description). The description adds context about the output but does not elaborate on parameter semantics beyond what the schema provides. It partially compensates for the missing limit description by implying its role in the list.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it tracks recent dataset updates on data.gov.ua (monitoring). It uses specific verb-resource (track updates) and distinguishes from siblings like 'explore_catalog' and 'find_datasets' by focusing on change monitoring.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions narrowing by topic or organization, implying when to filter. However, it does not provide explicit when-to-use or when-not-to-use guidance, nor does it name alternatives among the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explore_catalogA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions 'cheap on tokens' which is helpful, but does not explicitly state that the operation is read-only or idempotent. The description covers the behavioral scope adequately.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences plus a usage hint, with no redundant information. Every sentence adds value, and the structure is front-loaded with purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description explains the return type (aggregates, not datasets). It covers purpose, usage, and parameters. Could detail the response format (e.g., fields returned) but is sufficient for selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 80% with descriptions for 4 of 5 parameters. The description adds context (e.g., 'query' narrows topic, group_by options), but does not significantly enhance understanding beyond the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it provides aggregate catalog overview (count, publishers, categories/formats) and explicitly distinguishes itself from siblings by noting it does not return datasets themselves. The Ukrainian text gives concrete usage examples.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit use cases (e.g., 'who publishes most data about X') and implies when not to use (when you need actual datasets). It does not explicitly name sibling alternatives but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_datasetsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool returns a compact ranked list with title, manager, formats, freshness, link, and search-narrowing hints, which is transparent for a search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with one main sentence and additional detail, front-loading the primary purpose. It is well-structured but could be slightly more structured for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For an 8-parameter search tool with no output schema or annotations, the description explains the return format and hints. It lacks details on pagination or sorting but is otherwise complete for typical search usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is high (87.5%), so the schema already documents most parameters. The description adds context about search narrowing hints but does not significantly enhance parameter understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds datasets on data.gov.ua by topic, specifies the return format (ranked list with details and hints), and distinguishes it from siblings by instructing to use it for any 'find data about' query.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to use this for any 'find data about' query, providing clear context. It does not explicitly mention when not to use it, but the purpose is well-defined enough to infer appropriate use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inspect_datasetA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description discloses input flexibility (ID/slug or name with auto-search) and output fields. However, it lacks details on read-only nature, side effects, or error cases. Adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is 3-4 sentences in Ukrainian, front-loading the purpose and then listing included fields. No fluff, but slightly longer than necessary due to detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description covers input, output contents, and suggests next steps (use get_dataset_data). It is complete for a metadata inspection tool, though error handling is omitted.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a detailed description for the single parameter. The description adds value by explaining that a name triggers auto-search for best match, going beyond the schema's basic string definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool provides a detailed card of a dataset including description, manager, license, freshness, and resources. It distinguishes from siblings like find_datasets and get_dataset_data by focusing on inspection before data retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description specifies this tool is for inspection before using get_dataset_data, implying a workflow. It does not explicitly mention when not to use or list alternatives, but the context is clear enough for an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_dataset_dataA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses that it reads from DataStore or parses locally (CSV/JSON/XLSX), returns a preview with row estimate and full file link. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with the verb, no wasted words. Each sentence provides essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 parameters, no output schema, and no annotations, the description covers core functionality, automatic behavior, output components, and usage context. Could mention error handling or permissions, but adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions. Description adds context: automatic resource selection for 'dataset', priority for 'resource_id', and default for 'limit'. Adds value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves data (first rows + column schema) from a dataset or resource. It distinguishes itself from siblings like explore_catalog or filter_data by focusing on data preview and automatic best-resource selection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'This is your main tool for "show data"' and explains automatic resource selection and fallback behavior. It could compare to siblings more, but the guidance is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• filter_dataA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description mentions read-only SQL, indicating safe operation. Though no annotations are provided, it implies non-destructive behavior. Missing explicit statement about side effects or idempotency, but sufficient given the context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with core purpose, then crucial usage restriction. No unnecessary words. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, output schema, and 7 parameters, the description covers all needed context: what it does, constraints, and integration with sibling tools. Complete and actionable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Adds meaning beyond the schema by clarifying that filters are exact column=value mappings, and that SQL overrides filters. Provides an example. Schema coverage is high, but the description enhances understanding without redundancy.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to filter/aggregate rows of a structured resource using exact filters or read-only SQL. It specifies the resource type (DataStore-active) and differentiates from siblings by noting the prerequisite check via inspect_dataset.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use (DataStore-active resources) and when not to (other resources, use get_dataset_data instead). Provides actionable guidance to verify has_datastore via inspect_dataset, and names specific alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.