Server Quality Checklist

Tool Scores

• geocode_addressA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations available, the description carries the burden of behavioral disclosure. It usefully mentions the free nature, lack of API key, and data source (Census Bureau), but does not disclose rate limits, error behavior, or output format details beyond the listed geographies.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, using two short sentences and a source attribution. It front-loads the core action and adds only necessary context about cost and source, with no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with no output schema, the description adequately outlines the input and expected outputs (lat/long, geographies). It could be more complete by explaining potential failure modes or usage limits, but overall it is sufficient for basic invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully describes the 'address' parameter with an example, so the description adds little parameter-specific meaning. The mention of 'US street address' reinforces the schema but does not go beyond it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: converting a US street address into latitude/longitude and Census geographies. This specific verb+resource distinguishes it from sibling tools that operate on census tracts or zones rather than addresses.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly indicates this is for address-to-geography conversion, which contrasts with siblings like lookup_census_tract or check_flood_zone. It does not explicitly exclude other use cases or name alternatives, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_census_tractA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden of behavioral disclosure. It adds useful context such as 'Free, no API key' and the data source, and specifies the census year (2020). However, it does not disclose potential limitations like US-only coverage or behavior for coordinates outside any tract, which would be relevant for a geocoding tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences and front-loads the main purpose. Every piece of information ('2020 census tract', 'county and state', 'latitude/longitude', 'free', 'no API key', 'source') adds value without unnecessary verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with only two parameters and no output schema. The description covers the core purpose, return content, and access constraints. It could be considered slightly incomplete for not mentioning that census tracts are US-only or what happens for points outside the US, but overall it provides enough context for a straightforward lookup tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides full descriptions for both parameters (latitude and longitude in decimal degrees, WGS84), so schema coverage is 100%. The description adds minimal semantic value beyond what the schema provides, only referring to 'a latitude/longitude point', which is adequate given the high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Return') and the resource ('2020 census tract') along with additional context (county and state) and input (latitude/longitude). It effectively distinguishes itself from sibling tools like geocode_address (which likely goes from address to coordinates) and get_tract_demographics (which focuses on demographics for a given tract).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when given a coordinate and needing the census tract, and notes it is free and requires no API key. It does not explicitly mention when not to use it or name alternatives, but the context and sibling tool list make the appropriate use case clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_flood_zoneA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool performs a read-only lookup via 'Return', checks SFHA status, requires no API key, and cites the FEMA NFHL source. It does not cover error handling or data freshness, but for a simple point-query tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three terse sentences: the first states the core function, the second addresses access (free, no key), and the third names the data source. Every sentence adds value and the structure is front-loaded with the most important information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity tool with no output schema, the description sufficiently explains the returned concept (flood-hazard zone and SFHA eligibility) and provides access and source context. It lacks explicit output format details, but that is not critical for a simple lookup. The description is complete enough for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides full parameter descriptions (latitude/longitude in decimal degrees WGS84) with 100% coverage, so the baseline is 3. The description merely echoes 'latitude/longitude point' without adding any new parameter nuances, such as coordinate order or bounds.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Return') and identifies the resource ('FEMA flood-hazard zone') with clear scope ('for a latitude/longitude point, including whether it falls in a Special Flood Hazard Area'). This clearly distinguishes it from sibling tools like geocode_address and lookup_census_tract.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by stating the tool's purpose and noting that it is free with no API key, which helps the agent infer when to use it. However, it does not explicitly state situations to avoid or name alternative tools, so it misses the bar for a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_tract_demographicsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description covers behavioral traits directly. It discloses the API key requirement, provides a sign-up link, and mentions the default field set when 'fields' is omitted. While it does not describe error handling or output format, these are minor for a simple retriever; the key prerequisite and default behavior are clearly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the purpose, then provides an essential prerequisite, a field list, and a usage note—all in three short sentences. No unnecessary words; every sentence contributes value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with two parameters, the description covers the core functionality, required input, available fields, and default behavior. It does not explain return format or error cases, but given the simplicity and lack of an output schema, this is a minor gap. The description is sufficiently complete for most use cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with both parameters fully described. The description adds little beyond the schema: it lists the available fields (matching the enum) and mentions the default set, but the schema already documents the default. Thus, the description does not significantly enhance parameter understanding beyond structured data.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Return') and resource ('demographic indicators for an 11-digit census tract GEOID'), and clearly distinguishes the tool from siblings by focusing on demographic data from the ACS 5-Year. The title and first sentence work together to convey exactly what is provided.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description sets clear context: this tool returns demographic data for a tract GEOID. It also mentions a prerequisite (API key) and gives a link to obtain it. However, it does not explicitly discuss when to use this tool over siblings like lookup_census_tract or check_opportunity_zone, though the purpose is distinct enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_opportunity_zoneA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden. It discloses that the tool is free, requires no API key, and auto-resolves address/coordinate inputs to 2010 census tracts, plus the Round 1 limitation. It stops short of describing the return format or error behavior, but covers the key traits for a simple lookup.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose immediately. It includes all necessary context without redundancy, making it both concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-purpose lookup tool with 100% schema coverage, the description covers the main use case, input formats, resolution behavior, and cost/auth. It does not mention the response shape or whether multiple inputs are mutually exclusive, but these are minor omissions given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by making explicit that address, lat/lng, and tract_geoid are alternative input modes and that address/coordinates are resolved to 2010 tracts. This goes beyond the schema's individual parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to determine whether a location is in a designated Qualified Opportunity Zone (Round 1, TCJA 2017). It specifies the resource (OZ status) and the exact scope, distinguishing it from siblings like check_flood_zone and lookup_census_tract.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context on accepted inputs (address, lat/lng, or GEOID) and explains the automatic resolution to 2010 tracts. However, it does not explicitly mention when to use this tool over alternatives (e.g., check_flood_zone) or provide exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.