Server Quality Checklist

Tool Scores

• search_awardsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that roughly 30% of award notices have no dollar figure, resulting in `award_value` null and counted in `coverage.unvalued_rows`, and interprets this as a withheld amount rather than a free contract. It also instructs to read `notes` and pass on its content. This adds meaningful behavioral context about data quality and null handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short paragraphs: purpose, use cases and alternatives, and a data caveat. Every sentence adds value, and the most important info is front-loaded. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with fully described parameters and an output schema, the description covers purpose, when to use, exclusions, and a critical data nuance (missing award values). It is complete enough for an agent to decide when to invoke this tool and know what to expect in results, without needing to explain the full output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 100% coverage, with each of the 5 parameters described in detail. The description adds minimal parameter-specific insight beyond emphasizing that `supplier` is preferred for filtering by winner, which is already noted in the schema's `keywords` description. Baseline 3 applies because schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search awarded Canadian federal contracts' and specifies what info it returns: 'who won, for how much, when, and for which buyer.' It distinguishes from siblings by explicitly pointing to search_tenders and summarize_spend for different use cases.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this for a supplier's track record, for what similar work has gone for, or for "who won X".' and then gives exclusions: 'Not for opportunities still open: search_tenders covers notices accepting bids.' and 'For ranked totals rather than individual contracts, use summarize_spend.' This is clear when-to-use and alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_tenderA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the exact-match requirement, non-error return for unknown references, default withholding of buyer contact details, the need for the 'pii' scope, and the policy that `include_contact` should only be set when the user explicitly asks. This is thorough and actionable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short paragraphs: purpose, usage context, and privacy/param guidance. Every sentence carries necessary information, and the description is front-loaded with the core function. There is no fluff or repetition beyond what the schema already states.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, the description does not need to explain return structure. It covers purpose, usage, failure behavior, privacy, and permissions. It is fully complete for a fetch-by-reference tool, and the distinction from siblings makes it contextually robust.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds meaningful context beyond the schema by explaining that an unknown reference_number returns no rows rather than an error, and by clarifying the privacy rationale for include_contact ('Buyer contact details are personal data and are withheld by default'). This reinforces and enriches the schema, though some of it is redundant.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Fetch one open tender notice in full') with a precise resource and scope ('by its exact reference number'), and enumerates the returned contents ('complete description, dates, category, trade agreements and buyer'). It clearly distinguishes itself from sibling tools like search_tenders (search vs. exact fetch) and search_awards (different resource type).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells when to use the tool: 'Use after `search_tenders` has found a notice worth reading.' It also defines exclusions: 'This does not search — the reference number must match exactly, and an unknown one returns no rows rather than an error.' This directly guides the agent away from misuse.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• summarize_spendA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully carries the behavioral disclosure burden. It transparently explains the two major caveats: undisclosed awards are excluded (not counted as zero) and only one currency is summed per call, plus instructs the agent to repeat the `notes` field when reporting figures.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with purpose and examples, then delivers critical caveats in a compact second paragraph. Every sentence adds value; there is no filler or redundant restating of schema information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description covers all needed decision-making context: use cases, alternatives, data caveats, and reporting instructions. It fully prepares the agent to use the tool correctly without needing external clues.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already has full descriptions for all 5 parameters, so the baseline is 3. The description adds meaningful extra semantics, particularly around the `currency` parameter (source mixes CAD/USD/EUR with no conversion, default is CAD, amount never combined) and group_by behavior, elevating it above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies a concrete action: 'Rank total disclosed contract value by buyer, supplier or category,' immediately identifying the resource and capabilities. It also distinguishes itself from siblings by explicitly pointing to `search_awards` for individual contract detail.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides direct usage guidance with concrete example queries ('who spends most on X', 'which suppliers earn the most') and explicitly contrasts the tool with `search_awards` for 'individual contracts behind a total'. This gives clear when-to-use and when-not-to-use context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• export_reportA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does so excellently: it discloses the never-writes-on-first-call design, the 5-minute single-use handle tied to exact content, the export scope requirement, and the content limitation (no buyer contact details). This fully informs the agent about side effects and constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with a concise summary sentence, then expands with necessary behavioral details. Every sentence contributes unique value; the multi-paragraph structure is appropriate for the two-phase workflow and is not padded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's two-phase mutation complexity, no annotations, and an existing output schema, the description covers all needed context: purpose, side effects, permissions, prerequisite workflow, and failure behavior. It is complete enough for an agent to select and safely invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema description coverage is 100% (baseline 3), the description adds significant workflow semantics: confirm_handle duration and single-use behavior, the effect of changing format/notices on handle validity, and the handling of not-found tender refs. These go beyond the schema and materially improve invocation correctness.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description immediately states the tool writes a report of tender notices to a server file in markdown or CSV format. This verb+resource+format specification clearly distinguishes export_report from the read-only sibling tools (search, get, summarize).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides an explicit two-phase workflow: call with tender_refs to preview, then call again with confirm_handle to write. It also instructs the agent to let the user decide and never confirm on their behalf, and mentions the required export scope. This is actionable guidance beyond simply stating existence.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_tendersA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and delivers richly: it discloses ordering behavior (soonest-closing first, or relevance with keywords), the ~5% null notice_url rate with instruction to quote reference number instead, the requirement to pass on notes field content, keyword matching limitations (no synonyms, OR/NEAR treated as words), and region exclusion reporting. This is exceptional transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than average but every sentence earns its place: purpose, usage scenarios, sibling differentiation, ordering, URL null handling, notes field instructions, and keyword limitations. It is front-loaded with the most important use-case information and each subsequent sentence addresses a distinct aspect, making it dense but efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 optional params, output schema present, multiple siblings), the description is fully complete. It explains ordering, null URLs, notes propagation, keyword behavior, region caveats, and timezone handling. The presence of an output schema means return-value details are not needed, and the description covers all operational edge cases an agent would need.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with solid per-parameter descriptions, but the description goes well beyond: it explains keyword semantics ('has no idea about synonyms', operators are ordinary words), warns that region filtering excludes 126 notices and to mention the count, and clarifies closing_before has no timezone conversion. This adds meaningful semantic context that the schema alone does not provide.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource+scope: 'Search open Canadian federal tender notices — solicitations still accepting bids.' It clearly distinguishes itself from sibling search_awards by stating it covers open tenders, not awarded contracts, and even names the alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says when to use: for 'what can we bid on', 'which contracts are coming up', or finding work in a field or region. It also gives a clear exclusion: 'Not for contracts already awarded: search_awards covers who won what, for how much.' This is model guidance with named alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.