Server Quality Checklist

Tool Scores

• schedule_new_jobB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must fully disclose behavior. Mentions creation and scheduling but lacks details on side effects, authorization requirements, or constraints like overlapping times or customer existence validation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with key action and audience. No unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, no annotations, and four required parameters, the description is too sparse. Lacks details on return values, error conditions, or prerequisites (e.g., customer must exist). Incomplete for a scheduling tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%; description adds minimal value beyond schema—only notes customer_id comes from create_pro_customer. Baseline 3 appropriate as schema already documents parameters thoroughly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the verb 'create and schedule' and the resource 'trade job associated with an existing customer'. Differentiates from siblings like create_estimate or create_pro_customer by focusing on scheduling a job for an existing customer.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides context that it is used by AI dispatchers to book service windows, but does not explicitly state when to use versus alternatives or when not to use. Implied usage but no exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_pro_customerA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It describes a creation action but lacks details on side effects (e.g., duplicate handling), authorization requirements, or output behavior. This is insufficient for a write operation with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with no superfluous information. It is front-loaded with the core action and use case, earning its place efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for a simple creation tool with four parameters, but it lacks information about return values, error states, or any constraints. Without an output schema, additional detail on what the tool returns (e.g., customer ID) would enhance completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents each parameter. The description adds 'capture inbound caller details before scheduling a job' but does not provide additional meaning or usage nuances beyond what the schema already conveys.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool registers a new customer in the Housecall Pro CRM and specifies the use case for AI receptionists capturing inbound caller details before scheduling a job. This verb+resource pairing is specific and distinguishes it from sibling tools like create_estimate or schedule_new_job.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly mentions the tool is used by AI receptionists to capture inbound caller details before scheduling a job, providing clear context. While it doesn't explicitly state when not to use it or list alternatives, the context effectively implies its purpose relative to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_job_invoiceA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that it is gated behind an environment variable and returns an error if not enabled, and that it creates a billable customer-facing invoice. Without annotations, this provides useful behavioral context beyond just the action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, no redundancy. Front-loaded with the main action. The gating condition is relevant but could potentially be integrated more succinctly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description covers the core purpose, a key behavioral gate, and the nature of the invoice created. Missing details like success response or effects on job status but acceptable for a straightforward action.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with parameter description 'Active job ID to invoice'. The tool description reinforces 'active' but adds minimal meaning beyond the schema for the single parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states 'Post an official invoice against an active job,' clearly indicating the verb, resource, and context. It distinguishes from siblings like create_estimate or schedule_new_job which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use or alternatives. The description mentions a gating env var but does not specify situations where this tool is appropriate versus other invoicing or billing tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• submit_new_leadA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It only says 'Log a new...lead' without disclosing behavioral traits such as side effects, idempotency, permissions needed, or error handling. The description lacks transparency beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the core action, and every word is informative. No redundant or unnecessary text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 required parameters, no output schema), the description adequately covers its purpose and usage context. It does not need to explain return values. The description is sufficient for an AI agent to understand when and how to use it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the input schema already describes the parameters ('customer_id' as 'Associated customer profile ID' and 'description' as 'Product interest or issue detail'). The description does not add any additional meaning beyond the schema, so it meets the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Log a new marketing or inbound prospect lead in the Housecall Pro sales funnel.' It specifies the verb 'log' and the resource 'lead', and distinguishes from siblings like 'create_estimate' or 'schedule_new_job' by focusing on inbound marketing leads.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives context: 'Used by AI agents handling inbound chat or web form submissions.' This implies when to use, but does not explicitly state when not to use or provide alternatives. However, sibling tools like 'create_pro_customer' or 'generate_job_invoice' serve as implicit alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_estimateA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses the tool creates an estimate, implying mutation, but does not detail authorization needs, error states, idempotency, or side effects. Basic transparency is present, but deeper behavioral traits are missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, with no redundant words. It front-loads the primary verb and object ('Generate a pricing estimate or service proposal') and immediately follows with usage context. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with only two required parameters and no output schema, the description is fairly complete. It covers purpose and usage context. However, it lacks information about return values (e.g., estimate ID) and potential error conditions, which would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear descriptions for both parameters. The description adds little beyond the schema, simply restating that the note is a 'memo description.' It does not provide additional semantics, thus baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Generate a pricing estimate or service proposal for a customer.' It distinguishes from siblings like generate_job_invoice by specifying it is for quoting after diagnostic conversations, making it easy for an agent to select correctly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage context: 'Used by AI sales agents to quote work after diagnostic conversations.' This tells an agent when to use the tool, though it does not explicitly mention when not to use it or direct alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.