Server Quality Checklist

Tool Scores

• get_series_infoA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears full responsibility for behavioral disclosure. It only says 'Returns' which implies a read-only operation, but it omits information about authentication, error handling, rate limits, or exact response format. This is a significant gap for a tool with no annotation safety signals.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one sentence preceded by a clarifying rhetorical question, providing all necessary information without any fluff. It is concise, front-loaded, and effectively communicates the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (single parameter, no output schema), the description adequately names the return fields (series title, source table, coordinate). It lacks details on error behavior, but for a straightforward lookup tool, the provided context is nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents the vector parameter with type ('string' or 'number') and an example ('v41690973' or 41690973). The description adds little beyond repeating that it takes a vector ID. With schema coverage at 100%, the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool does: 'Returns the series title, source table, and coordinate for a vector ID.' This is a specific verb plus resource, and it distinguishes from sibling tools by focusing on decoding a vector ID rather than searching tables or fetching data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The informal question 'What is this vector?' implies use when you have a vector ID and want to identify its series. However, there is no explicit guidance on when not to use it or which sibling tool to use instead, leaving the context to be inferred.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_data_by_coordinateA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full responsibility for behavioral disclosure. It says 'Fetch observations' (implying read-only) but does not describe the return structure, format, error behavior, or any side effects. This minimal transparency is a significant gap for a tool with no output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly two sentences: the first states the purpose concisely, the second provides a illustrative example. No superfluous information, front-loaded with the primary verb, and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With three parameters and no output schema, the description covers the core concept and includes a helpful example, but it omits what the returned observations look like. It is reasonably complete for understanding how to invoke the tool, yet the absence of any description of the response structure leaves a gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All three parameters have schema descriptions, giving 100% coverage. The description adds an example that clarifies the semantics of member_ids, but it does not materially enhance understanding beyond the schema, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the action ('Fetch observations') and the resource ('table by coordinate'), with a concrete example (table 18100004, member_ids [2,2] = CPI, Canada, all-items). It also references get_table_metadata, distinguishing it from sibling get_data_by_vectors which likely uses a different query method.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context: users must obtain member IDs from get_table_metadata, and the example demonstrates correct usage. However, it does not explicitly contrast with alternatives like get_data_by_vectors or state exclusions, so it falls short of a full 'when to use vs. not' guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_data_by_vectorsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the 10-vector limit and the default for latest_n, which is useful. However, it does not explicitly state that the operation is read-only, mention any side effects, rate limits, or error behavior. 'Fetch' implies reading, but the disclosure is minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loads the core action, and packs a lot of useful info (limit, example, default, parameter options) without any fluff. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 parameters, no output schema, and no annotations, the description covers the main usage aspects: what a vector is, the limit, the parameter choice, and the default. It does not specify whether start_date and end_date must both be provided together, nor what the response format looks like. However, the description is adequate for a basic data-fetching tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds valuable semantics by clarifying the relationship between the date range and latest_n ('Give a date range, or latest_n'), the default value, and the meaning of vector IDs with an example. This goes beyond the individual property descriptions in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Fetch' and identifies the resource as 'observations' for 'Statistics Canada vector IDs.' It clearly explains what a vector is and distinguishes from sibling tools like get_data_by_coordinate (which uses coordinates) and get_series_info (which provides metadata). The example 'v41690973 = CPI all-items Canada' grounds the purpose concretely.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear usage instructions: 'Give a date range, or latest_n most recent periods (default 12).' This tells the agent how to structure the query. It does not explicitly mention when to use this tool over siblings, but the context (vector IDs) is clear enough to guide selection. Lacks explicit exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_tablesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It transparently discloses the tool's scope (search across all ~8,200 tables) and return behavior (product IDs for get_table_metadata). It does not mention nuances like result ordering or pagination, but for a read-only search tool this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first states the action and scope, the second explains the return value and next step. It is front-loaded, specific, and contains zero wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with no output schema, the description adequately covers what it searches, what it returns (product IDs), and how to proceed (get_table_metadata). Minor gaps like result ordering or match criteria are not essential for usage. Overall complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with both 'query' and 'limit' having clear descriptions including examples and bounds. The tool description adds no additional parameter context beyond what the schema already provides, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: keyword search across all ~8,200 Statistics Canada data tables, with topic examples. It also distinguishes from siblings by explicitly linking to get_table_metadata for further exploration, making it evident this is a discovery tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: first find table IDs via search, then use get_table_metadata. It provides context with examples (CPI, GDP, labour force) and a clear workflow reference. However, it does not explicitly state when NOT to use it or name alternative tools, leaving the differentiation implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_table_metadataA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavior. It transparently mentions that large dimensions are truncated, which is a key operational trait, and states that member IDs are returned for use with another tool. This adds significant context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise, consisting of just two sentences. It front-loads the purpose and delivers key operational details (truncation, filtering, and next-step usage) without any fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (3 parameters, no output schema, no annotations), the description covers the critical context: the output structure, truncation behavior, and how to use the results. It could arguably mention the return format explicitly, but it is sufficiently complete for an AI agent to understand and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents all three parameters with 100% coverage, so the baseline is 3. The description adds value by giving a concrete example for member_filter ('gasoline') and explaining the rationale for truncation (related to max_members), enriching the parameter semantics beyond the schema's definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to return the structure of a Statistics Canada table, specifically its dimensions and members with member IDs. This distinguishes it from sibling tools like get_data_by_coordinate, which retrieves data, and search_tables, which searches for tables.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides practical usage guidance by explaining the truncation behavior and advising to use member_filter (e.g., 'gasoline') for finding specific members. It also connects to get_data_by_coordinate, suggesting a workflow. However, it lacks explicit 'when not to use' or direct alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.