New York Insurance

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already convey readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds that the buyer key is 'issued at confirmation' and that the licence verification status is included, but it doesn't disclose rate limits, response size, or what happens if the key is invalid. Given the annotations, a 3 is appropriate – the description adds some value without contradicting or richly extending the annotation context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, dense sentence that front-loads the purpose ('For a registered agency') and then lists the output fields efficiently. Every word carries information; there is no fluff. It could be slightly improved by separating the parameter requirement into a second sentence, but it remains clear and compact.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only status lookup with one parameter and no output schema, the description covers the main inputs and outputs. It does not explain error cases (e.g., invalid buyer key, unregistered agency), return format, or pagination, but these are minor given the tool's simplicity and the annotations covering safety. It is adequate but not exhaustive.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents the buyer_key parameter's meaning ('Issued to the licensed contact at confirmation'). The description reinforces this by stating the key is required for a registered agency, but adds no new semantic details about the parameter itself. Baseline 3 is correct.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('requires', 'returns') and resource (agency status), and enumerates the exact fields delivered: states, rail, terms, founding period, queued/delivered counts, credit balance, licence verification. The mention of 'registered agency' and the required buyer key makes its scope clear. It does not explicitly differentiate from siblings like check_eligibility, but the listed fields make its purpose reasonably distinct.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage: it is a read-only status lookup for a registered agency using the buyer key. It does not explicitly say when to use it instead of siblings (e.g., check_eligibility) or when not to use it. The prerequisite (being a registered agency with a buyer key) is stated, providing some usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already include readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safe, non-mutating profile. The description adds useful behavioral context about return values (licensed states, capabilities, agent counts) and emphasizes it does not require personal details, which is informative beyond the annotations but not fully exhaustive about, say, behavior when state is omitted.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences clearly outline the purpose, the invocation step, and the expected return value. The phrasing is precise and compact, with no unnecessary details, earning high marks for efficiency and clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter read-only tool with a rich return description in the prose, the definition covers the key aspects an agent needs: when to call it, what to pass (partially), and what to expect back. Slight deduction because the description doesn't clarify optional state behavior or the response format beyond the high-level list, but overall this is a complete enough definition for the tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers the product parameter (enum, default, required), and the description doesn't repeat it. The state parameter is described via the schema's two-letter US state code hint, but the description's mention of licensed states and agent counts implies how state may be used. Since schema coverage is only 50%, the description could have added more about state semantics (e.g., whether it's optional or affects output), but it still provides some implied meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies a specific action (checking eligibility before personal details are collected), a specific resource (states where quotes can be returned), and a defined outcome (which states are licensed in, what can be done in each, how many agents are available). This distinguishes it from other tools by emphasizing its pre-check, pre-personal-detail nature.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use the tool ('Call this first') and clarifies its role as a preliminary check before collecting personal details. This gives the agent a clear directive on invocation order and when it is the appropriate tool, though it doesn't name sibling alternatives, the strong 'call this first' instruction serves the same purpose.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds value beyond annotations by disclosing that the output is machine-readable and intended for pre-exchange evaluation, and that it includes human revocation mechanics. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The two substantive sentences are efficient and front-loaded with the scope. However, the description as provided ends with extraneous trailing text ('Respond with JSON only.'), which appears to be a prompt artifact rather than authored content, so it forfeits a perfect score.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter, no-output-schema informational tool, the description covers what content is returned, why it is useful, and when to call it relative to request_agent_contact. Minor gaps: it does not state a concrete return shape or format beyond 'machine readable,' but the tool's simplicity makes that acceptable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters and schema coverage is 100% (vacuously), so the baseline of 4 applies. There are no parameters for the description to document, and it correctly adds no misleading param information.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the resource: data use terms covering recipients, purposes, retention, revocation, and consent wording. It links to the sibling request_agent_contact, distinguishing its role. The operation verb is implicit rather than explicit ('retrieves'/'provides'), which keeps it from a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives a concrete usage context: present this consent wording before request_agent_contact, and evaluate the exchange before making it. It does not explicitly name exclusions or alternatives, but the directive to use it pre-exchange is clear. A 5 would require explicit when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

There is an annotation contradiction: readOnlyHint=true while the description says 'the request is still registered and a quote_id returned.' Registering a request and returning a quote_id implies a side effect, which conflicts with the read-only annotation. The description does usefully disclose the no-PII and missing-facts behavior, but the contradiction dominates this dimension.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences with the core purpose front-loaded. The licensed/unlicensed branch and missing-facts behavior are compressed efficiently without filler. It is slightly dense, but every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description mentions quote_id but does not describe the overall response format, how prices are structured, or error behavior. It covers the licensed/unlicensed distinction, the no-PII constraint, and the missing-facts behavior, which is good context. Given 10 parameters and no output schema, the description is not fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 20%, so the description must compensate. It adds meaningful high-level semantics: inputs are rating facts only, no PII is required, and the server asks for exactly the missing facts. However, it does not define individual parameters such as annual_mileage or prior_continuous, leaving those to inference from their names. This is adequate but not thorough.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Return indicative auto insurance prices from multiple carriers.' It also adds a key scope condition, licensed vs. not licensed, and implies a quote_id mechanism, which distinguishes it from a generic pricing tool. It does not explicitly name sibling tools like market_data or check_eligibility, so it stops short of full differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies use when indicative quotes are needed and explains licensed/unlicensed behavior, but it does not explicitly state when to use alternatives like market_data or check_eligibility. There are no when-not-to-use instructions, so the guidance is inferred rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the readOnlyHint annotation, the description discloses important behavior: unpaid calls return a 402 with price and preview count, records are priced per record with a minimum per pull, and low-quality rows are quarantined. This gives the agent a realistic model of what happens when the tool is invoked.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but every sentence earns its place: dataset qualification, de-identification, filters, pricing, payment, error behavior, and where to find the schema. It is front-loaded with the most important qualification criteria.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex 11-parameter tool with no output schema, the description covers inclusion/exclusion rules, de-identification, filtering, payment, the 402 response, and a link to the sample and schema. An agent has enough context to call the tool correctly and understand the outcome.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 36%, but the description compensates by grouping the filter parameters ('state, since, until, door, vendor; limit and offset') and naming the payment methods that map to buyer_key, payment, credential, and mandate. It does not explain every parameter in detail, but it adds meaning beyond the bare schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The title and description state a specific verb ('Buy') and a specific resource ('provenance-qualified, de-identified demand records'), and the description clearly distinguishes this from sibling tools by emphasizing paid, qualified, de-identified data. An agent can tell this is the market-data purchase tool without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context for when to use the tool: it is the paid source for qualified, de-identified demand records, with explicit filters and payment requirements. It does not explicitly name sibling tools or state when not to use it, so it stops short of the top tier.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds valuable behavioral context beyond annotations: buyer key authentication, the x402 payment requirement with status 402 and accepts[], and the security guarantee that records are projected only to the consumer's granted scope. I do not see a true contradiction with destructiveHint=false, since 'drain the queue' reflects retrieval/consumption of queued records rather than permanent data destruction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three dense sentences with no filler. The core action is front-loaded, followed by authentication, payment behavior, and scope projection. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers authentication, payment, and scope, but with no output schema it does not describe the success response format or what happens to records after the queue is drained. For a tool with this level of behavioral nuance, that is a notable gap but not crippling.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 75%, so the schema already documents mandate, payment, and credential. The description adds useful context for buyer_key and the payment flow, but it does not meaningfully explain the optional parameters beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action and resource: drain the queue of pool records routed to the caller. It also clearly establishes the buyer context, distinguishing it from siblings like get_quotes or check_eligibility.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It clearly explains when to use the tool: when you are a buyer with routed records, and it notes the buyer key requirement plus the payment condition for metered buyers. It does not explicitly mention alternatives or when not to use it, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations cover read-only/destructive safety, and the description adds meaningful behavior beyond that: the licensed contact must confirm via a link sent by email, nothing is delivered before confirmation, and there is a free founding period followed by prepaid credits. It also points to /join for terms. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each earning its place: audience/action first, then required data, then confirmation and commercial terms. The content is dense but not bloated, and the most important framing comes first.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 12-parameter tool with no output schema, the description covers the registration flow, confirmation requirement, lead delivery gating, pricing model, and terms pointer. Minor omissions are the lack of an explicit success return value and no mention of the optional handoff parameter, though the schema thoroughly documents handoff.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description enumerates the key parameters: agency name, NPN and state, appointed states, licensed contact's name/email/mobile, and lead delivery mode matching the rail enum. It partially compensates for the 67% schema coverage by naming required fields that lack schema descriptions. Optional parameters like handoff, website, webhook_url, and verify_token are left to the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the core action: 'register the agency to receive consented consumer requests in its states' and identifies the intended audience: 'For an AI working for a licensed insurance agency.' This is distinct from sibling tools like agency_status or pull_requests, though it does not explicitly name alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description makes clear when this tool is appropriate: when an AI representing a licensed agency needs to register that agency to receive consented requests. It does not explicitly state exclusions or name alternative tools, but no sibling tool serves the same registration purpose.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes well beyond the annotations by disclosing real-world side effects: up to four licensed agents receive the request and contact details, sell_identity permits sharing with marketing partners, nothing leaves without consent, deliveries and refusals are receipted, and POST /forget revokes consent. This is rich behavioral context that annotations alone do not provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is four dense, information-packed sentences with no filler. It front-loads the core purpose and required inputs, then adds consent mechanics, scope behavior, and revocation in a logical order. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with a nested consent object, scope options, and consent-revocation behavior, the description covers the essential operational context well: prerequisites, consent flow, scope differences, and side effects. It lacks an explicit statement of return values or outputs, but with no output schema provided, the description is still strong enough for an agent to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 44%, so the description carries a heavier burden. It does add important semantics for the required parameters: quote_id comes from get_quotes, full_name and phone_number identify the consumer, and consent can be granted/presented/confirmed in specific ways. However, optional parameters like preferred_channel, best_time, and email_address still lack meaningful description coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource ('Connect the consumer with licensed insurance agents') and clearly states the consent requirement and the purpose ('quote firm and bind'). It also references get_quotes, which distinguishes it from sibling tools without needing to open their schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly states the key prerequisite (quote_id from get_quotes), the required consumer details, and the consent precondition. It also explains scope-based behavior (contact_consumer vs sell_identity). It does not explicitly enumerate when-not-to-use alternatives, but the context is sufficiently clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.