Server Quality Checklist

Tool Scores

• search_federal_contract_opportunitiesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses pricing ($0.004 per opportunity), that it runs on an Apify actor, and requires APIFY_TOKEN. This is good transparency, though it could mention error handling or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single paragraph that is front-loaded with the tool's purpose. It is concise and contains no wasted words, though it could be slightly more structured with bullet points for readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description should explain return values. It mentions some output fields (contracting-officer names, emails) but does not fully describe the structure of returned records. Given the tool's complexity (12 parameters), it is somewhat incomplete on outputs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (all 12 parameters have descriptions). The tool description adds value beyond schema by explaining the pricing implication of maxResults and recommending postedAfter for performance. It does not introduce new parameters but enriches understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches US federal contract opportunities from SAM.gov, listing specific notice types (RFPs, solicitations, etc.) and data included. It distinguishes itself from sibling tools (building permits, healthcare providers) by targeting federal contracts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for GovCon business development and lists filters but does not explicitly state when to use it over alternatives or when not to use it. It provides context but lacks direct exclusions or comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_building_permitsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses the tool runs on the user's Apify account requiring APIFY_TOKEN, pay-per-result pricing, and returns normalized permit records. Lacks details on error handling or rate limits but is transparent about costs and external dependence.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  4-sentence description, well-structured: purpose, returned fields, use case, technical details and pricing. Front-loaded with core purpose. The last sentence is dense with pricing details but remains understandable. Efficient for the information conveyed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, supported cities, return fields, cost model, and authentication requirement. Lacks explanation of error handling if APIFY_TOKEN missing, and output structure is not described (no output schema). For 7 optional parameters and paid actor, the main gaps are minor.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already has descriptions for all 7 parameters (100% coverage). The description adds value by explaining the use case (lead generation) and cost implications (e.g., maxResultsPerCity billable). Also adds recommended usage hint for issuedAfter. Supplements schema with decision-relevant context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches US building permits from specific cities, specifies normalized fields returned, and distinguishes from sibling tools which cover federal contracts and healthcare providers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides context for construction lead generation and market research, and mentions the paid Apify actor with pricing. Does not explicitly state when not to use or compare to alternatives, but sibling tools are clearly different.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_healthcare_providersA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries full burden. It discloses the tool runs a paid Apify actor on the user's account, requires APIFY_TOKEN, and explains billing ($0.002 per record). This covers safety and cost implications well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is somewhat long but structured with key info upfront (what it is, what it returns, filters). Every sentence adds value: purpose, returned data, filters, use cases, and billing details. No redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 9 parameters, no output schema, and no annotations, the description provides sufficient context: it lists return fields, explains all filter parameters, and covers cost. It lacks only explicit mention of pagination or error handling, but overall it is quite complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptive parameter definitions. The description adds value by explaining wildcard usage (e.g., '331*', 'mayo*') and billing implications of maxResults. This enhances the agent's understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches the official CMS NPPES NPI Registry for US healthcare providers and organizations, listing returned fields and filter criteria. It distinguishes itself from sibling tools (building permits, federal contracts) by its specific domain.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit use cases: 'healthcare sales lists, provider verification, and directory enrichment.' It does not specify when not to use this tool, but the clear purpose and unrelated siblings make this adequate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.