Server Quality Checklist

Tool Scores

• get_usaspending_awardsB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for behavioral disclosure. It adds value by specifying the return contents (award counts, obligations, top awards), which goes beyond a generic 'get data'. However, it omits details such as default limit handling, behavior when fiscal_year is omitted, or any potential limitations, leaving some ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loaded with the main purpose and followed by the key outputs. Every word earns its place, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with only three scalar parameters and no output schema, the description covers the core purpose and returns. However, it does not explain what happens if optional parameters like fiscal_year or limit are omitted, nor does it describe the structure of the returned awards data beyond a high-level list. There is no guidance on edge cases or integration with sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for all three parameters, so the baseline is 3. The description does not add new meaning beyond what the schema already provides; it merely restates the agency and fiscal year context. No extra clarification on defaults or parameter interactions is given.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves federal awards data for a specific agency and fiscal year, with a verb ('Get') and resource ('federal awards data'). It lists the main outputs, but does not explicitly differentiate from sibling tools like spending_by_category or search by recipient, so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for agency-level awards data but provides no explicit guidance on when to choose it over the sibling tools (e.g., spending_by_category, budgetary_resources). No alternatives or exclusions are mentioned, leaving the agent to infer usage solely from the tool name and description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_usaspending_spending_by_categoryB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states the core function and omits important details such as whether the response is a list of categories with totals, whether fiscal_year is required despite being optional in the schema, and any limitations on which award categories are included. No return format or aggregation behavior is described.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that directly states the tool's purpose. No unnecessary words or repetition, and the key information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description should explain what the returned spending breakdown looks like (e.g., category names and totals). It does not, nor does it clarify the optionality of fiscal_year. The tool is simple, but the description leaves significant gaps for an agent attempting to invoke it correctly without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for both parameters, so the schema already explains agency_code and fiscal_year. The description adds context by mentioning 'for an agency and fiscal year' but does not clarify that fiscal_year is optional, which could confuse an agent. Since the schema is strong, a baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: 'Get spending breakdown by award category' for an agency and fiscal year. It distinguishes itself from sibling tools like get_usaspending_awards (which likely lists individual awards) and get_usaspending_budgetary_resources (budget-level data) by focusing on category aggregation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used when you need spending totals per category for an agency and fiscal year, but it provides no explicit guidance on when to use this tool versus alternatives like get_usaspending_awards or search_usaspending_awards_by_recipient. There are no exclusions or prerequisites mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_usaspending_budgetary_resourcesA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only implies a read operation ('Get') but does not disclose output format, whether it returns aggregate or detailed records, data source specifics, or any access limitations. This leaves significant behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence, front-loaded with the verb and resource. It is efficient with no wasted words and immediately conveys the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low parameter count and no output schema, the description covers the essential intent but omits any details about the return structure or typical usage context. It is minimally sufficient but not enriched with additional contextual guidance that would help an agent select it confidently over siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and both parameters have clear descriptions with examples. The tool description does not add any extra meaning beyond the schema, so the baseline score of 3 applies. It neither clarifies parameter relationships nor provides additional format details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') and clearly identifies the resource ('budgetary resources and obligations') and the scope ('for an agency in a fiscal year'). It is distinct from sibling tools like get_usaspending_awards and get_usaspending_spending_by_category, which focus on different aspects of spending data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies this tool is for retrieving agency-level budget/obligation data for a given fiscal year, which differentiates it from the award- and category-focused sibling tools. However, it does not explicitly state when not to use it or name alternatives, falling short of a full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_usaspending_awards_by_recipientA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility for behavioral disclosure. It only says 'Search' and mentions filters, but does not disclose return format, pagination, rate limits, or read-only behavior. This is a significant gap for a tool with zero annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no wasted words. The first sentence front-loads the main action and scope, and the second succinctly adds a use case.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema is present, so the description should cover return values, but it does not. It adequately explains the tool's purpose and filters but omits behavioral details. For a straightforward search tool, it is minimally sufficient but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds a light grouping of filters ('time period and amount ranges') that maps to fiscal_year and min/max_amount, but does not add meaningful semantics beyond the schema's per-parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search for federal awards by recipient name' with a specific verb, resource, and key parameter. It distinguishes from sibling tools by focusing on recipient-name search and gives a concrete use case ('investigating specific companies or organizations').

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context ('Great for investigating specific companies or organizations') that implies when to use the tool. However, it does not explicitly mention alternative tools like get_usaspending_awards or state when not to use it, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_award_detailA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full disclosure burden. It conveys a read-only verification operation and explains the nuanced input behavior: a bare PIID is 'resolved via award search first.' It does not mention failure modes, but that is a minor gap for a single-record lookup.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three efficient sentences each earn their place: the main action in the first, the practical use case in the second, and input details in the third. The important verification purpose is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the key verification fields and both input formats, even with no output schema. Slightly more context about error handling or unavailable records could improve it, but it is largely sufficient for correct use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents the single parameter at 100% coverage. The description adds value by mapping the generated award ID to 'the contract_id Tango returns' and clarifying that a bare PIID must first be resolved, which is useful beyond the raw schema text.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Names a specific verb ('Get') and resource ('full FPDS record for a single award from USASpending'). The 'verification primitive' phrase and the listed competition-related fields clearly distinguish it from the sibling search/aggregation tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context: 'Use it to confirm that a filtered search actually returned what it claimed before a number goes in a memo.' It does not explicitly name alternatives or exclusions, but the when-to-use intent is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_reference_codeA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and discloses that this is a static lookup, requires no network and no API key. This is meaningful behavioral information, but it does not mention error handling or behavior for invalid/unknown codes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded purpose, then domains, then a pragmatic usage directive. There is no filler and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple, static lookup tool with no output schema, the description adequately covers what the tool does, the accepted domains, typical usage timing, and its local nature. The main omission is not describing the exact return format; however, the output scope feels obvious and the description still is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers 100% of parameters, so the baseline is 3. The description adds semantic value by contrasting 8A ('8(a) Competed') with 8AN ('8(a) Sole Source'), which helps the agent understand the meaning of the lookup before filtering or reporting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb (look up / list) with a clear resource (official FPDS descriptions for procurement reference codes) and enumerates the five accepted domains. It differentiates well from the sibling USASpending data-query tools, which are named around 'get'/'search' rather than local code lookup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells agents to use this tool before filtering or reporting on codes, and provides a concrete 8A vs 8AN example to illustrate why. It does not explicitly name sibling tools as alternatives or state when not to use it, so it falls just short of the highest benchmark.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• aggregate_contractsA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations were provided, so the description carries the full behavioral-disclosure burden. It exceeds that burden by disclosing upstream rate/performance implications ('one upstream query per code, ~4s each'), the date_type echo behavior, and the restrictions on metric 'count'. These are behavior traits an agent cannot infer from the schema alone.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description front-loads the core behavior, then gives a concrete example, then notes key behavioral caveats. Every sentence earns its place, and there is no filler, fluff, or duplicated schema content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 13-parameter tool with no annotations and no output schema, this description is quite complete: it explains when to use it, what the aggregation semantics are, and what performance/rate implications to expect. It could still be improved by explicitly describing the response/result shape, which the current description only partly implies.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds real semantic depth above the schema by explaining why group_by=set_aside has unique cost behavior, why count is only supported there, and that counts state their unit. It doesn't repeat every parameter, but it gives the context that makes parameters more usable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening sentence names the action ('Aggregate federal contract award...') and the resource (USASpending), and explicitly distinguishes this tool from row-returning tools with 'without pulling rows'. It also enumerates the supported group_by dimensions and metrics, making the tool's scope unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'This is the tool for questions like...' and gives a concrete query shape, which tells the agent exactly when to select it. It also clearly communicates the boundary for when not to use it: 'without pulling rows', which contrasts with the row-returning siblings, and restricts metric=count to set_aside group_bys.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.