Server Quality Checklist

Tool Scores

• get_scrape_resultsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint, idempotentHint, and openWorldHint. The description adds valuable non-obvious behavior: 'Does not spend credits — safe to call repeatedly,' which is crucial for polling. It also clarifies the two-phase nature (status check then fetch).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: first states purpose, second states cost/safety. Every word earns its place; no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and strong annotations, the description covers purpose, polling behavior, cost impact, and parameter origin. It does not describe the response format, but since no output schema exists, a mention of business records being fetched is reasonable. Minor gap on exact return structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with job_id described as 'Job id returned by start_scrape_job or scrape_google_maps.' The description adds no further parameter details, so baseline 3 applies. It does reinforce provenance, but that information already exists in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Check the status of a scrape job and fetch the business records once it is complete.' It uses a specific verb-resource pair and distinguishes itself from siblings like start_scrape_job (which initiates) and get_credits (which queries balance).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage after starting a job, especially through the parameter reference to 'start_scrape_job or scrape_google_maps.' It clearly conveys when to use (to check status and fetch results), but does not explicitly discuss exclusions or alternatives beyond the implied workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_creditsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and openWorldHint, which cover safety. The description adds behavioral context beyond annotations by noting 'Each scrape costs 2 credits' and 'Free to call,' clarifying the tool's cost and side-effect profile. This is useful additional context, though it does not detail rate limits or error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary purpose, then adds one cost detail and a 'free to call' note. Every sentence earns its place with no waste or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema, safe by annotations), the description provides sufficient context: what it does, it's free, and the credit cost per scrape. It is complete for an agent to understand when and why to use it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the input schema has 100% coverage (no properties). Per the rubric, when there are no parameters, a baseline of 4 is appropriate. The description does not need to explain parameters it does not have.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's function with a specific verb ('Get') and resource ('remaining gmapsscraper.io credit balance'). It is distinct from siblings like start_scrape_job or get_scrape_results, which focus on scraping operations, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states it is 'Free to call,' implying it can be used without cost or side effects, which gives clear context for when to invoke it. It does not explicitly name alternatives or exclusions, but its purpose in relation to sibling scraping tools is obvious, so clear context is provided but no explicit when-not or alternative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scrape_google_mapsA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds substantial behavioral context beyond annotations: cost per call, non-idempotency with an explicit 'do not retry' warning, blocking duration of 30-120s, and user-confirmation requirement. This aligns with the idempotentHint=false annotation and enhances the agent's understanding.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two purposeful sentences. The first states the action and output fields; the second packs cost, non-idempotency, confirmation, keyword cost, blocking time, and keyword advice. Every phrase earns its place and is front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 params, no output schema), the description covers the return fields, cost, blocking, and retry behavior. It does not mention the async fallback (job id) that appears in the max_wait_seconds schema, but that is covered structurally. Slight gap in explicit sibling differentiation prevents a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value for the 'keywords' parameter (cost behavior, specificity guidance, example) and clarifies that 'email' extraction is conditional via the email parameter. This goes slightly beyond the schema without overburdening the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Search Google Maps and return business leads' followed by a concrete list of returned data fields. This clearly distinguishes the tool from siblings like start_scrape_job and get_scrape_results, which handle async portions of the workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context: costs 2 credits, must confirm with user, supports multiple related keywords at the same cost, and advises specific keyword formatting. However, it does not explicitly mention when to use alternative tools (e.g., for async scenarios), though the blocking behavior and max_wait_seconds schema hint at it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_scrape_jobA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses credit cost per call, non-idempotent behavior ('NOT idempotent, do not retry on your own'), and the need to confirm with the user before spending credits. These go beyond the annotations (readOnlyHint=false, idempotentHint=false) by adding financial and guidance context. It also warns about retries, which is valuable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences, each adding distinct information: purpose, cost/idempotency, user confirmation, and sibling comparison. No redundant phrasing or unnecessary details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool involves async job submission with cost implications, and the description covers all actionable aspects: the async behavior, credits, non-idempotence, user confirmation, job id return, and polling step. Given no output schema, the 'Returns a job id' line provides the critical return information, and the sibling guidance completes the picture.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  While the schema fully documents all 5 parameters, the description adds the economic detail that 'Multiple related keywords cost the same 2 credits,' which is not in the schema. It also hints at the `radius` parameter with 'large areas,' but the schema already covers per-parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the verb 'Submit' with a clear resource ('Google Maps scrape job') and specifies asynchronous execution ('without waiting for it to finish'), distinguishing it from the synchronous `scrape_google_maps` sibling. It also mentions returning a job id, clarifying the tool's core function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states 'Use this instead of scrape_google_maps for large areas or when the user wants to continue working meanwhile,' providing a direct alternative comparison. Also indicates the follow-up flow with `get_scrape_results`, making the usage context clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.