Server Quality Checklist

Tool Scores

• get_crime_dataA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint=false, so the safety profile is covered. The description adds meaningful behavioral context: quota consumption (1 call per key), the 90-day rolling window, 1000ft radius, and the free test exception. These operational details go beyond annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but front-loaded with the most important info (what it returns) before usage guidance. Every sentence earns its place. Slightly long but each clause conveys distinct value; could be trimmed slightly but nothing is filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is an output schema present, so return values are handled. The description covers inputs, constraints, exclusions, quota/auth, and a test path. For a moderately complex tool with 3 params and good schema coverage, this is near-complete. Minor gaps: no mention of response format details, but output schema covers that.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 3 parameters. The description adds the key constraint that address and lat+lng are mutually exclusive and the format example. This is useful but modest supplement; baseline 3 is appropriate given full schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description has a specific verb+resource ('Get Crime Data & Safety Score') and clearly states what it returns: a letter score, per-crime-type breakdown against national average, and incident history within 1000ft over a 90-day window. It distinguishes from siblings by explicitly naming them and what they cover.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Excellent when/when-not guidance. States use case ('is this address/neighborhood safe?'), explicit exclusions (demographics, income, rent, schools via get_neighborhood_data; offenders via search_offenders), input constraints (address XOR lat+lng, US only), and quota/API key requirements including a free test address.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_neighborhood_dataA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, openWorldHint=true, idempotentHint=true, destructiveHint=false. The description adds meaningful context beyond annotations: API key requirement, quota cost (1 call), the free test address exception, US-only location constraint, and the nullable 'neighborhood' field behavior. It does not specify the exact return format/nested structure, but the output schema exists to cover that.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is information-dense with no wasted sentences, front-loading the purpose and data returned. It packs many facts (scope, exclusions, key, quota, test address, null behavior) efficiently into about 90 words. Slightly long but every clause earns its place; no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the multiple data categories returned, output schema presence, and comprehensive annotations, the description covers most important usage constraints. It documents exclusion (crime), scope (US), auth (API key), cost (quota), test address, and address/latlng exclusivity. It does not elaborate on the output structure, but the output schema handles that. Minor gap: no explicit mention of error cases or what happens with invalid locations beyond the null neighborhood field.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters clearly. The description adds the 'Pass either address or lat+lng, not both' constraint, which is useful, but does not add format details beyond what the schema provides. The schema's param descriptions explain lat/lng/address adequately, so the description's marginal value is real but limited.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs and resources: 'Profile of the neighborhood... demographics, median household income, home values, rent by bedroom count, education attainment, cost-of-living indices, climate and natural-disaster risk.' It clearly distinguishes from siblings by explicitly stating it does NOT return crime data and pointing to get_crime_data. The scope (US locations, one address/coordinate) is narrowly defined.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use it ('when the question is about who lives in an area or what it costs to live there') and when NOT to ('Do NOT use it for crime or safety scores' with a named alternative get_crime_data). It documents the address-vs-latlng exclusivity constraint, US-only scope, API key requirement, quota consumption, and the free test address.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_api_usageA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety profile is well covered. The description adds genuine behavioral value beyond annotations: the 'never consumes quota' property (a strong side-effect disclosure), the 'always describes calling key, never another account' scoping constraint, and the confirmation-of-limit intent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with the core purpose and scoping constraint first, then usage triggers. Efficient and well organized. Minor deduction only because the second sentence slightly nests multiple clauses (reset-time confirmation + upgrade suggestion) that could be streamlined.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-arg, fully-annotated, output-schema-bearing tool, the description is remarkably complete: it covers return content (usage, quota, rate limits, overage), scope, invocation triggers, and a distinctive non-quota-consuming behavior. Loses one point only for not explicitly naming the sibling-contrast, though the different domain of siblings makes that unnecessary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Zero parameters — the schema is an empty object. The description explicitly addresses this: 'Takes no arguments — it always describes the calling key.' This preemptively eliminates any agent confusion about parameterization, which is the only semantically relevant point for a no-arg tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Specific verb 'Report' plus clearly enumerated content (plan, usage, quota, reset times, rate limit, overage) bound to 'the DoorProfit API key authenticating this session.' Explicitly distinguishes from sibling tools that query crime/neighborhood/offender data — a very different purpose domain.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives explicit when-to-use triggers: when the user asks how many calls remain, and after any other tool returns QUOTA_EXCEEDED or DAILY_LIMIT_EXCEEDED to confirm limits before suggesting an upgrade. Sets clear expectations about the scope (never another account) and adds a distinct proof-point: the only tool that never consumes quota.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_offendersA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes well beyond the annotations by disclosing pagination behavior, the per-page API key requirement, quota consumption of 1 call per page, radius defaults/caps, and precedence rules. The read-only, idempotent, and non-destructive annotations align with a search operation, and the description adds substantial behavioral detail (quota costs, pagination) that the annotations don't cover.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three dense, purposeful sentences that front-load the core purpose, then mode details, then usage/exclusion guidance, then pagination/quota mechanics. Every sentence earns its place; no filler or repetition of schema content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 12-parameter tool with a rich output schema, the description is remarkably complete: it covers mode selection, parameter grouping constraints, precedence, defaults, caps, pagination, and API key/quota implications. The output schema presumably documents the result structure including distance, so the description need not restate return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so all 12 parameters are documented. The description adds meaningful semantic value by explaining the radius default/cap, the 'three filters alone not accepted' rule (city/state/zipcode), location precedence, and name-mode requirements — clarifying how parameters are grouped beyond what individual schema descriptions provide.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches US registered offenders and immediately distinguishes two distinct modes (location vs name). The verb+resource are specific, and it differentiates from sibling get_crime_data by explicitly noting it's for registered-offender questions only.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Exceptionally clear usage guidance: it explains the two modes (location vs name), states required conditions for each (name needs at least one of first/last/dob; location precedence over name), and explicitly says when NOT to use it ('for an overall safety grade use get_crime_data instead'). This names the alternative tool directly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.