Server Quality Checklist

Tool Scores

• boc_get_group_observationsA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds no new behavioral details beyond the example, missing context like pagination or data freshness.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with a front-loaded verb phrase and a helpful example. No unnecessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With annotations covering safety, an output schema present, and a clear description of core functionality plus example, the tool is well-specified. Minor missing context about group source or data limits, but overall sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers two of three parameters (group_name, response_format) with descriptions. The description's example implies 'recent' usage but doesn't explicitly define it, leaving a gap. Coverage is adequate but not enhanced.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool retrieves observations for all series in a group, with a concrete example illustrating use. It distinguishes from sibling tools like boc_get_observations which target individual series.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description notes efficiency ('in one call') and gives an example, but does not explicitly state when to use this tool versus siblings like boc_get_observations or boc_list_series.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• boc_list_groupsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint, openWorldHint, idempotentHint, and destructiveHint=false. The description adds context about groups being curated bundles, but does not reveal behavioral traits beyond what the annotations convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two well-structured sentences, front-loaded with the core purpose, no fluff. Concise and informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, pagination parameters, and search functionality, the description is complete. It explains what groups are and how to use them, providing sufficient context for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description mentions feeding a group name into another tool, which implies the output can be used as input, but does not add specific parameter-level details beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists data groups, gives a concrete example ('FX_RATES_DAILY'), and explains how to use the output with a sibling tool, distinguishing itself from other tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly guides usage by suggesting to feed the result into 'boc_get_group_observations', but does not explicitly state when to use or not use this tool compared to alternatives like 'boc_list_series'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• boc_get_series_infoA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false. Description adds minimal behavioral context (e.g., no mention of rate limits or authentication). It does not contradict annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first states purpose, second gives usage guidance. No unnecessary words or redundancy. Well-structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, the description adequately covers purpose and usage. It does not address error handling or naming conventions, but these are minor omissions for a simple read-only tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%; both parameters (name, response_format) are well-described in the schema. Description adds no extra parameter details beyond what is already present.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Get the label and description metadata for a single series,' using a specific verb and resource. It distinguishes from sibling tools like boc_get_observations and boc_list_series, which handle observations or listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises use 'before pulling its observations,' providing clear context and implicit exclusions. Though alternatives are not named, the guidance is actionable and aligns with sibling tool purposes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• boc_get_observationsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint false. Description adds value by explaining that the tool returns actual data points and handles multiple series and date ranges, consistent with annotations. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise: three sentences and an example. The purpose is front-loaded, and every sentence adds value. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present, the description need not explain return values. It covers the core functionality, parameter usage, and an example, fully capturing the tool's purpose for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. Description adds context beyond schema descriptions, such as explaining that 'recent' is for latest values and giving an explicit example of how to use series_names with recent. This enhances comprehension.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Retrieve observations (the actual data points) for one or more series,' which is a specific verb+resource. It positions itself as 'the core tool,' implicitly distinguishing it from sibling tools that list series or get metadata.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides usage patterns (date range vs recent, multiple series), but does not explicitly state when to use this tool versus its siblings (e.g., boc_list_series or boc_get_group_observations). The example helps but lacks direct differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• boc_list_seriesA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, destructiveHint. Description adds that it returns a paginated list with name, label, description, providing useful detail beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, each essential: purpose, usage order, search/next step, return structure. No redundant words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's purpose and the presence of output schema, description fully covers discovery workflow, pagination, and integration with sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with good descriptions. Description adds strategic context: mentions filtering to narrow catalogue and pagination, but does not repeat schema details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool lists/search Bank of Canada data series. It distinguishes from siblings by indicating this is the first step to discover series names.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs when to use ('first when you do not already know a series name'), how to filter with 'search', and the next step ('feed a returned name into boc_get_observations').

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.