Server Quality Checklist

Tool Scores

• undata_get_country_profileB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses the tool is read-only by nature ('returns the latest available value') and that it's a snapshot/overview rather than exhaustive. However, it doesn't clarify how many indicators are returned, whether the output is structured per-indicator, or any pagination/limits behavior. The period-scoping behavior (latest value in requested period) is a useful disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficient: three sentences conveying purpose, behavior, and parameter format. It's front-loaded with the main purpose. There's no wasted text, though a brief note about the year parameter format would have been valuable within the same sentence budget.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there's no output schema and no annotations, the description carries the full burden. It covers purpose, behavior (latest value semantics), and country-code format. Gaps remain: no description of the return structure, no year format guidance, and no relationship to sibling data-fetching tools. For a tool with 2 simple params and no output schema, it's moderately complete but leaves the agent guessing about response shape and year semantics.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter documentation. It explains the 'country' parameter format (ISO-3 codes with examples). However, the 'year' parameter is undocumented in both description and schema beyond a nullable string with default null. The description partially compensates by implying year scoping, but this is vague and provides no format guidance (e.g., 'YYYY' vs range).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool does: get a comprehensive economic and social snapshot for a country using WDI. It distinguishes itself from siblings like undata_get_wdi_data by noting it returns 'the latest available value for each indicator in the requested period' for a quick overview. However, it doesn't explicitly contrast with sibling tools like undata_compare_countries, so differentiation is partial.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context ('useful for a quick overview') but provides no exclusions or when-not-to-use guidance. It mentions ISO-3 country codes must be used, which is helpful. However, it doesn't differentiate when to prefer this over undata_get_wdi_data or undata_compare_countries, which are close siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• undata_get_mdg_dataB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It describes the return format as organized by series code over time, which is helpful. However, it doesn't disclose rate limits, whether year filtering defaults to the full range, or behavior for invalid country codes. Given no annotations and a read-style operation, the description provides moderate behavioral context but leaves gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, all informative and front-loaded with the core purpose. The ISO-3 code guidance is immediately actionable. No filler or repetition. Could be more efficient but is reasonably compact.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 3 parameters with 0% schema coverage, no annotations, and no output schema. The description covers the tool's scope, return structure, and country code format, but leaves start_year/end_year semantics unexplained and provides no guidance on what happens with invalid input or how the series structure is nested. Adequate for a moderate-complexity read tool but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains the 'country' parameter format (ISO-3 codes with examples) but says nothing about start_year and end_year semantics (e.g., whether they must be valid years within 2000-2020, whether they're inclusive, or string vs int format). Since 2 of 3 parameters are completely unaddressed, the description only partially compensates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets MDG data for a country, lists content categories (poverty, education, health, gender, environment), time span (2000-2020), and output structure (series code by time). It distinguishes itself from siblings via the MDG indicator focus, though it doesn't explicitly name sibling alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context on what data is included and explicitly instructs on ISO-3 country codes with examples. However, it doesn't contrast this with sibling tools like undata_get_wdi_data (World Development Indicators) or undata_compare_countries, so an agent may not know when to prefer this over those alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• undata_compare_countriesA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral burden. The description discloses the return structure (organized by series → country → year) which is useful behavioral context. However, it doesn't describe data limits, response size, pagination, error handling, or whether missing data for a country is handled gracefully. It's a read-only query but that's implied rather than stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each delivering meaningful information: what it does, how to call it, and what the return looks like. No filler or repetition. It could arguably drop the last clause but the return structure note is genuinely useful for an agent predicting output shape. Well front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 4 parameters, no output schema, and no annotations, the description does a reasonable job but has gaps. It explains the countries parameter (ISO-3 codes), series (WDI codes), and year range, and gives the output structure. However, for a comparative analysis tool across countries, it doesn't address cardinality/limits (max countries per call), nor whether missing series for certain countries returns nulls or errors. Decent but could be richer.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for undocumented parameters. The description adds some meaning: it mentions ISO-3 codes for 'countries', WDI series codes for 'series', and a year range for start/end_year. However, it doesn't clarify the year format (string vs number), whether end_year is inclusive, or the null-handling behavior of the optional parameters beyond the defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear purpose: compare World Development Indicators across multiple countries. It specifies the verb 'compare', the resource ('World Development Indicators'), and the multi-country scoping, which distinguishes it from the sibling tools dealing with single-country data (get_country_profile) or listing/searching (list_dataflows, search_wdi_series). It doesn't explicitly differentiate from get_wdi_data but the comparative framing is distinct enough.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides concrete usage guidance: use ISO-3 country codes, optionally restrict to WDI series codes and a year range. It explains the comparison is organized by series → country → year. It gives clear context for when to use it (comparative analysis across countries) but doesn't explicitly state 'when not to use' or name alternative tools, so not a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• undata_queryA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that it 'Returns raw parsed rows' and warns to 'limit to avoid huge responses,' which is useful. However, it doesn't disclose the query's performance characteristics, potential rate limits, failure modes, or how errors surface. The limit/disclosure is helpful but incomplete for a generic tool covering arbitrary dataflows.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is reasonably compact given the complexity of explaining SDMX dimension keys. The examples are front-loaded and directly instructive. It packs a lot of signal into a moderate length. Could be slightly tighter, but the examples are valuable and earn their space.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a generic tool with no output schema and no annotations, the description does a good job explaining core mechanics. However, it doesn't cover error handling, whether invalid dimension keys fail fast, the meaning of the returned 'raw parsed rows' structure, or how to explore available dataflows (though undata_list_dataflows exists as a sibling). The examples cover common use patterns well but leave edge cases undocumented.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does so well: it explains the key parameter format in detail ('dimensions separated by dots, blank means all values'), shows flow_ref usage with example values, and demonstrates start_year/end_year usage. It explains the key parameter thoroughly but doesn't separately elaborate on limit beyond mentioning responses can get huge. Still, strong compensation for the 0% coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it's a 'Generic SDMX query for any UNdata dataflow' with a specific verb+resource. It provides concrete usage examples and explains the dimension key format. It distinguishes itself from siblings by explicitly saying to use it 'when the specific WDI or MDG tools don't cover your dataflow,' which differentiates it from undata_get_wdi_data and undata_get_mdg_data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance on when to use this tool ('Use when the specific WDI or MDG tools don't cover your dataflow') and provides three concrete examples with flow_ref/key pairs covering different scenarios. It doesn't name alternative sibling tools explicitly but points to the category of WDI/MDG specific tools. Could be improved by naming alternatives directly, hence not a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• undata_search_wdi_seriesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that the tool returns 'matching series codes and sample values' and explains the output feeds another tool, adding value. However, it doesn't disclose response format details, pagination, result limits, or whether default year/country parameters affect results—gaps given zero annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no filler. The description front-loads the core purpose and includes practical usage tips (example keywords, downstream tool reference). Not quite a 5 because it could reorganize to put the output-chaining instruction more prominently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 3 parameters at 0% schema coverage, no output schema, and no annotations—so the description bears full weight. It covers purpose, examples, and downstream chaining, which is solid. But the unexplained year/country parameters and lack of clarity about what 'sample values' means leave gaps for an agent needing complete operational understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains keyword semantics well (search term for desired topic) and notes the series_code output purpose. However, the year and country parameters are entirely unexplained in both the schema and description—the agent won't know why defaults of '2022' and 'PER' matter for a search operation, which is a meaningful gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource combination ('Search for WDI series codes by keyword') and clarifies the goal is to discover available indicators. It explicitly distinguishes its role from siblings by explaining how it feeds into undata_get_wdi_data, a unique differentiator not present in other sibling definitions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use it (to discover which indicators are available) and gives concrete example keywords. It names the downstream tool (undata_get_wdi_data) and instructs the agent to use the series_code output in the series_filter parameter, but doesn't explicitly state when NOT to use it or name alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• undata_get_wdi_dataA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the return organization (series → year → value) which is useful. However, it doesn't mention pagination, data availability limits, whether invalid country codes error out, or rate-limit behavior. The description adds value but lacks deeper behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three compact sentences with zero waste. Each sentence earns its place: purpose, usage format, and return shape. The ISO-3 examples are efficient and instantly actionable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a data-retrieval tool with zero annotations and no output schema, the description covers core purpose and parameter hints well. However, it omits what a series code looks like, whether results are limited/paginated, year range constraints, and error handling for invalid inputs. Adequate but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for 4 undocumented parameters. It does explain the country parameter (ISO-3 codes with examples) and mentions series_filter and year range conceptually, but doesn't detail the series_filter value format or how start_year/end_year interrelate. Partial compensation for the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Specific verb+resource+scope: 'Get World Development Indicators (WDI) data from UNdata' clearly identifies source and content type. It describes return shape (economic and social indicators for a country across years) and differentiates from siblings by focusing on the WDI dataset specifically.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides concrete usage context including ISO-3 country code format with examples ('PER', 'USA', 'BRA', 'COL', 'CHN') and optional filtering by series codes or year range. Doesn't explicitly exclude when not to use it versus siblings like undata_query or undata_get_mdg_data, but the ISO-3 guidance is genuinely useful.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• undata_list_dataflowsA
  4.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It does disclose the return payload shape (dataset IDs, names, agencies, descriptions), which is useful. However, it doesn't mention list size, pagination, rate limits, or whether the list is exhaustive/filtered. Given a simple read/list operation with good payload disclosure, a 3 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences, no filler. Each sentence earns its place: what it returns, what to do with the output, and how it fits into the tool workflow. The front-loaded purpose makes it immediately scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter discovery tool with no output schema and simple complexity, the description fully covers what's needed: what's returned and how to use the results. No gaps remain for an agent to guess at call semantics or return interpretation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and schema coverage is 100%, so there are no parameters to explain. The description adds value by explaining what the returned values mean and how to use a returned identifier downstream, compensating for the absence of any structured parameter documentation. The 0-param baseline of 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description has a specific verb+resource ('List all available UNdata SDMX dataflows'), clearly states what's returned (dataset IDs, names, agencies, descriptions), and explicitly positions it as the discovery entry point distinct from sibling tools that consume the 'id'. This strongly differentiates it from the other undata_* sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this first to discover what data is available, then use the 'id' in other tools like undata_get_wdi_data or undata_query.' This gives clear guidance on when to invoke this tool (as a first step) and names concrete alternative/next-step tools, providing strong usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.