Server Quality Checklist

Tool Scores

• get_rba_tableB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, so the base transparency is high. The description adds 'normalised response shape', which offers some insight into output format but does not reveal behavioral traits like error handling or default behavior. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no wasted words. It is front-loaded but uses jargon ('Expert/source-native', 'normalised') which may reduce clarity. Could be more straightforward.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 6 parameters, an output schema, and is used for retrieving statistical tables, the description is too sparse. It lacks explanation of pagination, error handling, or typical use cases. The annotation-rich structure mitigates some gaps, but completeness is low.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with detailed descriptions for all 6 parameters. The description adds no additional meaning beyond what the schema provides (e.g., 'table_id' is described as non-empty dataset or table id). Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses 'retrieval' as the verb and specifies 'RBA statistical table' as the resource, clearly indicating the tool's function. It distinguishes from siblings like 'list_rba_tables' (list vs get) and 'get_abs_data' (different source). The phrase 'normalised response shape' hints at output structure, but the purpose is clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives like 'list_rba_tables' for discovery or other statistical retrieval tools. No exclusions or context are provided. The term 'Expert/source-native' implies specialized use but lacks clarity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_abs_dataB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint, destructiveHint, idempotentHint, and openWorldHint. The description adds 'normalised response shape' but does not disclose behavioral traits like pagination, rate limits, or error handling beyond what annotations offer.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short sentence, which is concise. However, the phrase 'expert/source-native' may be ambiguous and lacks clarity for an agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 7 parameters and an output schema, the description is minimal. It does not explain how parameters interact or provide usage context. Adequate but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters. The description does not add extra meaning to the parameters beyond what is in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states it retrieves ABS data in a normalised SDMX format, clearly identifying the resource (ABS) and action (retrieval). However, it does not explicitly differentiate it from sibling tools like get_apra_data or get_rba_table, though the name itself helps.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. There is no mention of prerequisites, exclusions, or comparisons with sibling tools like get_economic_series or describe_dataset.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_datasetA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already disclose readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering safety. The description adds valuable behavioral context by emphasizing 'source-native' and 'without hiding native IDs', which signals output representation nuances beyond the structured annotations. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded with the verb and resource scope, and every word contributes meaning. No filler or redundant phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The existence of an output schema covers return-value expectations. The main gap is the unresolved ambiguity between table_id and identifier, which the description does not address. For a moderately complex tool with four parameters, a bit more clarification on parameter usage would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter (source, table_id, identifier, include_structure) already described in the schema. The tool description adds no parameter-specific meaning and does not clarify the ambiguous relationship between table_id and identifier, so the baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies a specific verb ('Describe') and resource ('dataset'), scoped to ABS, RBA, and APRA sources, and adds a distinctive detail ('without hiding native IDs') that differentiates it from sibling tools. It doesn't enumerate what the description includes, which keeps it from a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit guidance on when to use this tool versus alternatives like get_abs_dataset_structure or get_rba_table. The description only states what it does, not the context for using it or exclusions. The agent is left without clear selection criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_derived_seriesA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint, destructiveHint, idempotentHint, openWorldHint. The description adds that date bounds accept specific formats and are normalized to frequency, offering meaningful behavioral context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: first states purpose, second explains date bounds. No wasted words, front-loaded with key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With output schema present, the description covers the main purpose and date handling. It does not mention the last_n parameter's truncated behavior, which is in schema but missing from description. Mostly complete for a tool with these features.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameters are well-documented. The description adds a clarification about date format normalization, but overall does not significantly enhance what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'read-only retrieval for the narrow transparent derived series layer', specifying the verb and resource. However, it does not explicitly differentiate from siblings like get_economic_series, limiting its clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. The description implies usage for derived series retrieval but provides no contextual or exclusionary information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_release_eventsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, which cover safety and behavior. The description adds minimal behavioral context beyond 'source-aware,' which is more of a feature than a behavioral trait. It does not contradict the annotations, and it does not add significant operational details such as pagination or return behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the primary action and target resource. Every word serves a purpose, with no filler or redundant phrasing. It is appropriately sized for a tool with self-documenting schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, rich annotations, and fully described parameters, the description is largely adequate. The main gap is that it does not clarify what 'release-pulse events' are, but the schema and output schema likely cover operational details. Overall, it is sufficiently complete for an agent to understand its core purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides 100% parameter description coverage, including details on source enum values, date formats, and defaults. The description itself adds no parameter-specific information, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as listing release events, with a specific focus on 'source-aware release calendar or release-pulse events.' This verb+resource structure distinguishes it from sibling tools like list_catalogue and list_economic_concepts. However, the term 'release-pulse events' is somewhat ambiguous and could benefit from elaboration.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used for listing release events, but it does not explicitly state when to use it versus alternatives, nor does it mention any exclusions or prerequisites. The usage context is implied rather than clearly articulated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_rba_tablesA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint, destructiveHint, idempotentHint, and openWorldHint. Description adds no behavioral context beyond deprecation status, which is standard and does not contradict annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no extraneous words, front-loaded with deprecation notice.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a deprecated alias tool with comprehensive annotations and full schema coverage, the description is adequate. It clearly tells the agent to use the alternative, satisfying completeness for its limited purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameters are fully described in the schema. Description does not add any parameter-specific meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states it's a deprecated compatibility alias and directs to list_catalogue(source="rba"), but does not explicitly describe its own functionality. Sibling differentiation is implicit via the alternative recommendation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises to prefer list_catalogue(source="rba") instead, providing clear guidance on when to use this tool vs alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_datasetsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, etc. The description adds context about curated sources but no further behavioral details like pagination or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single concise sentence that is front-loaded, contains no filler, and effectively communicates the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, the description doesn't need to explain return values. However, it lacks details like result limits or matching behavior, making it minimally adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for both parameters. The tool description does not add new information beyond the schema, meriting the baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies the verb 'search' and the resource 'curated ABS, RBA, and APRA economic datasets,' clearly distinguishing it from sibling tools like get_abs_data or list_catalogue.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for discovery but does not explicitly state when to use this tool versus alternatives or provide exclusion criteria. No guidance on when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_apra_dataA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare the tool is read-only and idempotent. The description adds that it retrieves public XLSX files and accepts only curated IDs, providing moderate additional context without contradicting annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two sentences. The first sentence states the core purpose, and the second adds a critical constraint. Almost zero waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the rich input schema and existing output schema, the description adequately covers key context: it retrieves APRA public XLSX data using curated IDs. It could mention error handling or output format more explicitly, but overall it is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with detailed parameter descriptions. The tool description adds little beyond confirming the curated ID requirement, so it meets the baseline 3 for high-coverage schemas.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves APRA public XLSX publications, specifying it is for curated official IDs only, distinguishing it from sibling tools that handle ABS or RBA data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions only curated APRA publication IDs are accepted, implying when to use, but does not explicitly state when not to use or compare to alternatives like describe_dataset or search_datasets.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_abs_dataset_structureA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint; the description adds context by specifying that the output includes 'dimensions and codelists', which goes beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with no redundant words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With one well-described parameter and an output schema (not shown but present), the description is complete for this simple retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with a clear 'Non-empty dataset or table id.' The description does not add further parameter semantics, but the schema already provides sufficient context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'ABS SDMX dataset dimensions and codelists', which distinguishes it from siblings that fetch data or list catalogues.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use this tool versus alternatives, but given its specificity and idempotent/read-only annotations, usage is implied when structural metadata is needed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_economic_conceptsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint, idempotentHint, openWorldHint, covering safety. The description adds context by specifying the relationship to get_economic_series, which is useful beyond annotations. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys essential information without any wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists and annotations are rich, the description is sufficient to convey the tool's purpose and relationship. It could mention that it is a read operation, but annotations already cover that.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline 3 is appropriate. The description does not add any parameter-specific meaning beyond the schema; it only restates the purpose.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists 'analyst-friendly semantic economic concepts' that are accepted by get_economic_series. The verb 'List' and resource are specific, and it distinguishes from siblings by linking to a specific sibling tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage before get_economic_series by stating concepts are accepted by that tool, but it does not explicitly state when to use, when not to use, or mention alternatives. Siblings like list_catalogue are not addressed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_catalogueA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true. The description adds minimal behavioral context beyond stating it lists curated entries. Since annotations are comprehensive, the description's contribution is adequate but not extra.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with two sentences front-loading the core purpose and filtering options. Every sentence adds value without redundancy, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 optional parameters, full schema descriptions, an output schema, and comprehensive annotations, the description provides sufficient context. It covers what the tool does, how to filter, and its relationship to a sibling tool, leaving no gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with detailed descriptions for each parameter. The tool description does not add additional semantics beyond what the schema already provides, fitting the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists curated entries from ABS, RBA, and APRA, with optional filters. It distinguishes itself from the sibling search_datasets by noting it is an unranked complement, making the purpose specific and clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions optional filtering by source, category, or tag, and explicitly positions itself as a complement to search_datasets, implying when to use this tool vs the search alternative. It lacks explicit when-not scenarios but provides adequate context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_latest_observationsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true. The description adds context about being a convenience wrapper, that it returns the latest observations, and that metadata.truncated indicates dropped older observations. No contradictions; it enhances the behavioral picture.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the core purpose ('Source-aware convenience wrapper for the latest observations'), and each subsequent sentence adds essential usage guidance without redundancy. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (not shown but indicated) and 100% schema coverage, the description is complete enough. It explains the tool's role among 14 sibling tools and provides necessary disambiguation. The only minor gap is not detailing the output structure, but the output schema handles that.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description does not add significant new meaning beyond what is already in the input schema's parameter descriptions. It mentions the count parameter's effect on truncation, but the schema already describes that. Thus, minimal added value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with 'Source-aware convenience wrapper for the latest observations', using a specific verb and resource. It immediately distinguishes itself from the sibling tool get_economic_series by explaining when to use that alternative instead. The purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells the agent to prefer get_economic_series(concept=...) for a single curated indicator and to use series_ids to narrow a broad dataset instead of returning every series. This provides clear when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_top_observationsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, openWorld, idempotent, and non-destructive hints, lowering the burden. The description adds useful behavioral context: it is a 'convenience wrapper' and warns that without narrowing, it may return every series in the dataset. No contradictions with annotations. Slightly more detail could be provided about return shape, but overall good.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: the first sentence states purpose, and the second provides usage alternatives. No redundant phrases; every sentence contributes. It is well-structured and appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 11 parameters and an output schema, the schema handles details. The description provides the essential conceptual context (wrapper, highest/lowest, source-aware) and high-level usage guidance. It could explicitly note that it aggregates across sources, but the schema's source enum already covers that. Overall complete enough for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description references series_ids for narrowing, but parameter semantics are mostly handled by the schema's detailed per-parameter descriptions. The description adds marginal value beyond the schema, so a 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: a 'source-aware convenience wrapper for highest or lowest numeric observations.' It uses a specific verb ('get'), resource ('top observations'), and explicitly differentiates from siblings by mentioning prefer get_economic_series for single indicators. This is a clear and distinguishing purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance: use this tool for highest/lowest observations, but prefer get_economic_series(concept=...) for single curated indicators. It also advises passing series_ids=[...] to narrow a broad dataset. This clearly states when to use and when to use alternatives, exceeding basic guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_economic_seriesA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, and non-destructive behavior. Description adds value by explaining date bound normalization and the metadata.truncated flag when using last_n, without contradicting annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two front-loaded sentences convey purpose, usage, and date format details efficiently without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 8 parameters, full schema coverage, and an output schema, the description covers key behavioral nuances and usage context. Minor gaps exist (e.g., error cases), but overall sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. Description adds meaning beyond schema by listing acceptable date formats and explaining the rationale for include_observation_dimensions defaulting to false.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves 'curated ABS/RBA economic concepts' and positions itself as 'Preferred analyst-facing retrieval tool', distinguishing it from siblings like list_economic_concepts for discovery.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use list_economic_concepts for discovery' and defines acceptable date formats, providing clear when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.