Server Quality Checklist

Tool Scores

• list_attachmentsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It proactively explains a common edge case: an empty list is a real state and returns zero, not an error. This adds genuine transparency beyond a simple 'list attachments' statement, though it does not cover authentication or other behaviors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no wasted words. The first sentence states the core function, and the second clarifies an important behavioral nuance. It is front-loaded with the action and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple listing tool with fully documented schema parameters and no output schema, the description adequately covers what the tool does and the notable empty-list behavior. It could mention usage relative to fetch_attachment, but that gap is more about usage guidelines. Overall it is sufficiently complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of parameters with descriptions, so the baseline is 3. The description does not add any additional parameter-level meaning; it only references the notion of 'one notice', which maps to solicitationNumber but adds no new detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns 'downloadable attachment URLs for one notice', specifying both the action (returns) and the resource (attachment URLs for a specific notice). This distinguishes it from siblings like fetch_attachment (which likely downloads a single attachment) and search_solicitations (which searches rather than lists for a known notice).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for a single known notice ('for one notice') but does not explicitly contrast with alternatives or state when not to use it. Sibling tool names provide context, but the description itself lacks direct usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_solicitationA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It explicitly discloses the abstention behavior when the number is not found within the window, and explains the rationale (distinguishing 'not in window' from 'does not exist'). This is valuable context beyond the schema, though it does not mention error formats or return details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The first sentence states the core action; the second adds an important behavioral caveat. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has three required parameters with full schema coverage and no output schema. The description covers the main purpose and a critical edge case (abstention). It implies the return value ('retrieves a single notice'), which is acceptable. Missing explicit return format is a minor gap, but the description is sufficiently complete for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the parameter descriptions already document all three fields. The description repeats the concept of a 'posted-date window' but adds no new syntax, defaults, or relationships beyond what the schema provides. Baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Retrieves' and the resource 'a single notice' by its solicitation number within a posted-date window. This distinguishes it from siblings like search_solicitations (which presumably does broad searches) and fetch_attachment (which handles attachments).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes the usage context clear: this is for retrieving a single notice when you have the exact solicitation number and a specific date window. It does not explicitly list exclusions or alternatives, but the context is unambiguous enough that an agent can infer when to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_solicitationsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavior. It does so effectively by adding non-obvious details: required date format and span limit, return of normalized notices with attachment links, and a warning that the returned count differs from totalRecords across all pages. This is valuable transparency, though it omits error behavior or pagination semantics beyond the count.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long, front-loaded with the core purpose, followed by constraints and then return/count behavior. Every sentence earns its place; no filler or redundant repetition of schema info.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 9 parameters and no output schema, the description covers the essential operational context: purpose, mandatory date window restrictions, return format, and a pagination count caveat. It does not detail how filters like limit/offset interact with the count, but those are covered by the schema descriptions, so the overall package is quite complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds value by stating that postedFrom/postedTo are required and at most one year apart—a rule not defined in the schema. It also hints at output semantics ('normalised notices'), which complements the schema's parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource+scope: 'Searches federal contract opportunities in a posted-date window.' This clearly distinguishes it from sibling tools like get_solicitation (retrieves a specific one) and attachment tools. The purpose is unambiguous and immediately identifiable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool (search within a posted-date window) and states hard requirements (postedFrom and postedTo required, MM/dd/yyyy, at most one year apart). It does not explicitly compare with alternative sibling tools, but the usage constraints and scenario are clear, so only a slight deduction for lacking explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• server_statusA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool reports mode and data source, implying a read-only operation, but it does not explicitly state that there are no side effects or describe the output format. This is adequate but not rich behavioral detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action and usage cue. There is zero waste or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the essential aspects of a simple diagnostic tool: what it reports and when to call it. It could elaborate on what 'mode' means or what constitutes 'surprising results,' but for a tool with no parameters and no output schema, it is substantially complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description cannot add parameter-level meaning. Per the baseline for zero-parameter tools, a score of 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Reports') and clearly defines the resource (server mode and data source). It distinguishes itself from sibling tools by focusing on server state rather than data retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: 'Call this first when results look surprising.' This gives a clear, actionable usage guideline, which is especially valuable for a diagnostic tool with no obvious alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetch_attachmentA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries the full burden. It discloses exact return behavior: returns UTF-8 text for text documents, and reports content type and byte length for binary formats instead of guessing or decoding. This is unusually transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two focused sentences, front-loaded with the main action. Every sentence adds value – the first states the operation and source, the second explains output handling. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description fully explains return behavior. The single parameter is well-defined in the schema, and the link to list_attachments provides necessary context for when this tool is used. Nothing important is missing for a simple fetch operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (the 'url' parameter is described as 'A resource link from list_attachments'). The description reinforces this but adds no new semantics beyond the schema, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Downloads a single attachment by its resource URL' – a specific verb, resource, and source. This distinguishes it from siblings like list_attachments, which lists attachments, and get_solicitation, which fetches a different entity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'as returned by list_attachments' explicitly indicates the prerequisite workflow. It doesn't name when not to use alternatives, but the context is clear and there is no competing download tool among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.