Server Quality Checklist

Tool Scores

• company_financial_historyA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the transparency burden. It discloses two key behaviors: deduplication to the most recent filing per period, and the inclusion of form type, filing date, and verify_url in each record. This is solid but does not mention potential errors, rate limits, or data ordering.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is succinct, with three short paragraphs that directly convey purpose, deduplication logic, and record contents. No redundant or vague phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, the description adequately explains what each record contains (form type, filing date, verify_url) and the deduplication rule. It lacks details on sorting, error handling, or metric value units, but the core context is covered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All four parameters have descriptions in the schema (100% coverage), so the baseline is 3. The tool description adds no new parameter-specific meaning beyond what the schema already states; it only reinforces the single-company scope implicitly via 'One company'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: retrieving a single company's reported history of a financial metric. It is distinct from sibling tools (e.g., search_federal_contracts) by focusing on company financials, and the verb+resource structure is specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide explicit guidance on when to use this tool over alternatives. It implies single-company scope but never states conditions like 'use when you need historical financial data' or contrasts with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_expiring_contractsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that each result includes days_until_expiry and the incumbent, that usaspending.gov cannot filter by end date, and that the tool walks end-date-sorted results and takes a few seconds. It also signals a read-only operation through 'Find'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is about 90 words, front-loaded with the core purpose, then provides market context, result fields, and a performance note in separate short paragraphs. Every sentence adds useful context for selection and expectation-setting.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 6 optional parameters and no output schema, but the description names the key result fields (days_until_expiry, current incumbent) and explains latency and data-source constraints. It is complete enough for a read-only search tool with richly documented params, though a full result-shape list would be even safer.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter already well-documented including defaults, ranges, and intent. The description reinforces the 'window' concept and output fields, but does not need to add parameter-level detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Find federal contracts expiring soon' which clearly specifies the resource (expiring federal contracts) and action (find). The mention of 'recompete opportunities' distinguishes this from the general sibling search_federal_contracts by emphasizing the specific use case.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides strong context: federal contracts whose performance period ends must usually be re-competed, and incumbents are beatable. This explains why an agent would choose this tool, though it does not explicitly name alternative tools or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• screen_public_companiesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses a key behavioral trait: 'Every record carries a verify_url to the filing the figure came from.' It also establishes the scope (all US public companies) and implies read-only screening. It does not mention rate limits or error handling, but for a read-oriented tool these are not critical. The description adds meaningful context beyond what annotations would provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: the first sentence states the purpose, the second paragraph gives concrete use cases, and the third adds one valuable detail (verify_url). It is front-loaded and every sentence earns its place. No wasted words or repetition of schema information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (7 params, no output schema), the description is fairly complete. It explains the tool's purpose, gives examples, and hints at output structure ('Every record carries a verify_url'). It doesn't detail all return fields, but that might be implied by the metric and ranking. The description is adequate for an agent to understand when and what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of parameters with descriptions, so the baseline is 3. The tool description does not add extra parameter semantics beyond what's already in the schema. The examples like 'revenue over $100 billion' reiterate the metric and period parameters but do not add new meaning. Therefore, it does not exceed the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Rank every US public company by a financial metric.' This is a specific verb+resource. It also distinguishes from siblings by emphasizing cross-company comparisons ('this compares all of them at once') and contrasts with SEC.gov/EDGAR. This differentiates it from company_financial_history, which likely focuses on individual company data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context on when to use it: 'Answers questions sec.gov structurally cannot: which companies had revenue over $100 billion last year...' It implies this is for cross-company screening. However, it does not explicitly mention when not to use it or name alternatives like company_financial_history for single-company history, so it stops short of explicit exclusion. This earns a 4, not a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_federal_contractsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations supplied, the description carries the behavioral disclosure burden. It discloses that results are limited to already-awarded contracts and that they are sorted 'largest dollar value first,' which goes beyond the schema. It does not mention pagination, output fields, or rate limits, but for a search/read tool the key behavioral traits are adequately surfaced.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: the first sentence states the tool's purpose, the second adds result ordering and use cases, and the final sentence points to an alternative. Every sentence earns its place with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has six optional, well-documented parameters and no output schema, so the description needs to convey scope and return behavior. It covers scope, sort order, and common use cases, but it does not enumerate the contract fields returned. Still, for selection and invocation, the description is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides thorough descriptions for all six parameters (e.g., 'Free-text search across award descriptions and recipient names'), so the description does not need to repeat them. It adds minimal parameter-level value beyond schema, but the schema coverage is 100%, meeting the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Search awarded US federal government contracts' and immediately distinguishes it from expiring-contract searches with 'Use this to see who is winning federal work.' It names specific use cases and references the sibling find_expiring_contracts, so an agent can tell this tool from alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is given on when to use this tool: for seeing who is winning federal work, what an agency spends, or a company's contract history. It also gives an explicit exclusion: 'For contracts coming up for re-bid, use find_expiring_contracts instead.' This clearly orients selection among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.