Server Quality Checklist

Tool Scores

• census_trade_get_top_partnersA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare read-only, idempotent, non-destructive behavior. The description adds context by explaining that the tool fetches the full country breakdown and sorts it because the API doesn't sort, and details the return shape (country, value, percentage share, grand total). This is useful behavioral context beyond what annotations provide and does not contradict them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with a clear one-sentence purpose, and the Examples/Don't use sections are effective. However, the Args section duplicates the schema's detailed parameter descriptions, making the description longer than necessary without adding proportional value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite lacking an output schema, the description details what is returned: ranked list with country, USD value, percentage share, grand total, and count. It also addresses the sorting workaround and provides multiple examples covering optional filters and time formats. For a read-only tool with rich schema annotations, this is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with detailed descriptions for all 9 parameters, so the baseline is 3. The description's Args section largely repeats schema information but adds concrete examples like filters={"E_COMMODITY":"2709*"} and default value_field selection. This adds minor value but doesn't significantly compensate beyond the schema's existing detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the top N countries ranked by U.S. export or import value for a given period and optionally a commodity. It distinguishes itself from sibling tools by explaining that the Census API doesn't sort results, so the tool fetches the full breakdown and sorts. The 'Don't use when' example further separates it from census_trade_get_trade_balance.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit 'Use when' examples for ranking export markets and importing crude oil, and an explicit 'Don't use when' for trade balance calculations, naming the alternative census_trade_get_trade_balance. It also points to census_trade_list_datasets for dataset selection, giving clear context on when to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• census_trade_lookup_country_codeA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already establish read-only, idempotent, non-destructive behavior, and the description adds context beyond that: the API filters by numeric CTY_CODE rather than country name, CTY_NAME cannot be used as a filter, and return values include code, name, and ISO alpha-2 details. It also flags special naming for North/South Korea. This is valuable behavioral transparency, though it doesn't mention rate limits or match-failure behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with intro, Args, Returns, and Examples sections, and it front-loads the core purpose and sibling relationship. It is somewhat longer than strictly necessary because the Args section repeats schema properties, but every other part contributes useful context such as API filtering behavior and usage examples.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple lookup tool with no output schema, the description is complete: it explains what the tool returns, the special case of country groupings, naming caveats, and how to chain the result into sibling tools. It covers when to use, when not to use, and gives concrete examples, so an agent can select and invoke it correctly without additional information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents all three parameters with defaults, enums, and constraints. The description's Args section largely restates this information, though the examples ('korea', 'vietnam', 'european union') and the note about partial name matching add modest practical meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'looks up' Census Bureau Schedule C CTY_CODE, including country groupings, and explicitly links this to filtering census_trade_query_exports/imports by country. This specific verb+resource framing and the mention of sibling tools distinguishes it from the query/list/dataset siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit use guidance: 'Use this tool to translate a country name into the code you need', plus concrete 'Use when' examples for Vietnam and European Union. It also gives an exclusion: 'Don't use when: You already have the CTY_CODE - go straight to census_trade_query_exports/imports', naming the alternative tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• census_trade_get_dataset_variablesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, establishing a safe read operation. The description adds valuable context beyond this: it notes the Census API rejects unknown variable names with a 400 error, and that valid variables vary by dataset (e.g., SITC only on the sitc dataset). This failure-mode information helps the agent anticipate user needs and error conditions without contradicting any annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized into distinct sections: purpose, usage, args, returns, and examples. Every sentence serves a purpose, the opening is front-loaded with the core verb and resource, and the examples provide concrete use cases. Despite being longer than average, it avoids redundancy and is scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there is no output schema, the description appropriately details the return structure ('For each variable - its name, human-readable label, whether it's required, and its type'). It also covers the full lifecycle: when to use, how to invoke, what to expect, and when not to use. The tool is fully self-contained and contextually complete for an agent navigating a complex API.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter (direction, dataset, response_format) having a full description and enum values. The description's Args section somewhat duplicates this information, but it adds a small amount of context, such as the note to 'see census_trade_list_datasets for the full list' and the default value for response_format. Since the schema already carries the heavy load, the description adds marginal value, keeping this at the baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'List every valid Census API variable (field) name for a specific dataset + trade direction.' This clearly distinguishes it from sibling tools like census_trade_query_exports/imports and census_trade_list_datasets. The explicit mention of listing fields rather than querying data or listing datasets removes any ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs when to use the tool ('Use this before calling census_trade_query_exports/imports when you're unsure which variable names are valid'), and provides a clear 'Don't use when' case with a named alternative (use census_trade_list_datasets instead). This exceeds typical guidance by giving both positive and negative usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• census_trade_list_datasetsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnly, idempotent, and non-destructive behavior, so the description's burden is lower. It adds valuable context that each dataset covers the same underlying monthly trade data (2010-present) and outlines the return structure (short code, full name, description, level of detail, accepted parameters). This goes beyond the annotations, though it doesn't discuss any side effects or potential pitfalls (none likely for a list operation).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the purpose, then flows into usage guidance, return format, and examples. Every sentence earns its place; the use/don't-use examples are particularly informative and not redundant. It's appropriately sized for a tool that needs to guide dataset selection.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with no output schema, the description fully covers what the agent needs to know: what the list contains, the structure of each entry, and how to use the results with sibling tools (e.g., the short code is used as the 'dataset' parameter in census_trade_query_exports/imports). It also gives helpful context about the commodity classifications and geography options.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema coverage is 100% (no properties). The description explicitly notes 'Args: none,' which is sufficient. Per the calibration, 0 params baseline is 4; there is nothing more to add.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('List'), a concrete resource ('the 9 commodity classification / geography datasets available in the Census International Trade Data API'), and scope ('for both exports and imports'). This clearly distinguishes it from siblings that query specific data, retrieve variables, or compute balances.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use guidance ('Use this first when you're not sure which dataset to query'), concrete example use cases ('What trade datasets are available?'), and an explicit exclusion ('Don't use when: You already know the dataset code you need - go straight to census_trade_query_exports/imports'). This is exemplary usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• census_trade_query_exportsA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, openWorldHint=true, idempotentHint=true, destructiveHint=false. The description adds significant behavioral context beyond these: API timeouts on large requests, descriptive text fields requiring code fields or the API errors, zero-row responses not being errors, and results not sorted by value. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-structured with clear headings (overview, Args, Best practices, Examples) and front-loaded purpose. Every section contributes value: usage guidance, behavioral constraints, and practical examples. It is longer than ideal but appropriate for a complex tool with 10 parameters and no output schema, and it avoids redundancy with the schema by providing contextual guidance rather than repeating enum values.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 10 parameters and no output schema, the description is exceptionally complete. It explains the return format (markdown/json), what rows contain, time range coverage, filter semantics, aggregation levels, error conditions, and provides multiple concrete examples. It also addresses edge cases (zero rows, large queries) and references sibling tools for complementary functionality. No significant gaps found.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema alone documents all parameters. The description adds meaning by explaining parameter relationships (e.g., time vs year+months alternatives), showing example values for filters with wildcard behavior, clarifying limit truncation client-side, and providing dataset-specific context (e.g., statehs vs porths). This goes beyond what the schema states, though much of the schema is already detailed, so the incremental value is solid but not exhaustive.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Query monthly U.S. export statistics (January 2010-present) from the Census International Trade Data API' and identifies it as the 'general-purpose tool' for export data. It distinguishes itself from siblings by explicitly listing what it is for (by commodity, geography, measures) and contrasting with import/trade balance/top partner tools, giving a specific verb-resource-scope combination.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit 'Use when' and 'Don't use when' examples that name alternative tools (census_trade_query_imports, census_trade_get_trade_balance, census_trade_get_top_partners). It also includes best practices from the Census API User Guide, such as preferring narrow queries and using get_dataset_variables to check valid parameters, which effectively guides selection between this and sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• census_trade_query_importsA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (readOnlyHint=true, idempotentHint=true), the description discloses important behaviors: the API times out on large requests, descriptive text fields require code fields or the API errors, results are NOT sorted by value, and zero rows may simply mean no trade. These are non-obvious traits not captured in annotations, adding significant transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-structured with paragraphs, bullet points, and examples. It front-loads the core purpose and uses sections for best practices and use cases. Some repetition exists with schema descriptions (e.g., limit behavior, descriptive field requirement), which could be trimmed, but overall each section earns its place for a complex 10-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description covers return format (markdown or JSON, rows containing requested fields plus time), common pitfalls, and dataset options comprehensively. For a complex tool with 10 parameters, the description covers query construction, filtering, aggregation levels, and error interpretation, making it complete enough for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds extra semantic value by explaining that import value fields start with GEN_/CON_ rather than ALL_ (export-only), and by clarifying dataset-specific behavior (e.g., statehs/statenaics/porths digit restrictions). However, many parameter details are already in the schema, so the added value is supplementary rather than essential.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening sentence states a specific verb+resource: 'Query monthly U.S. import statistics (January 2010-present) from the Census International Trade Data API.' It clearly differentiates from sibling tools by focusing on imports and explicitly noting 'Don't use when: You need export data - use census_trade_query_exports.' The scope is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance with 'Use when' examples and 'Don't use when' alternatives, naming sibling tools (census_trade_query_exports, census_trade_get_trade_balance, census_trade_get_top_partners). It also includes best practices for narrowing queries to avoid API timeouts, which is actionable when/then guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• census_trade_get_trade_balanceA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that it sums values across periods, computes per-country plus combined totals, and explains the sign convention for surplus/deficit. Also warns about invalid time format ('2024' must be expressed as 'from 2024-01 to 2024-12'), which is beyond the annotations' read-only/idempotent hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with a clear lead sentence, args list, returns section, and examples. Despite length, every section adds necessary context—no filler. Front-loaded purpose and usage guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 7-parameter tool with no output schema, the description includes expected return structure, parameter formats, examples for common use cases, and explicit exclusions. Covers the complexity well.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all parameters with descriptions (100% coverage), so baseline is 3. The description adds value by giving concrete examples (e.g., country codes for Canada/Mexico, hs_code='87') and clarifying the correct time format with an invalid example. However, some parameter descriptions repeat schema text, so not a full 5.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  First sentence states the exact operation: compute U.S. trade balance (exports minus imports) for specified countries and time period. Explicitly differentiates from sibling tools by noting it combines two queries, making it distinct from census_trade_query_exports/imports.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit use cases with 'Use when' examples and a 'Don't use when' that directs to sibling tools (census_trade_query_exports/imports). This gives clear guidance on when to select this workflow tool over alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.