Server Quality Checklist

Tool Scores

• escalate_to_vendorA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does well by describing the multi-step process (updates ticket status, triggers workflow, sends messages) and the override capability. However, it doesn't mention potential side effects like whether this action is reversible, permission requirements, rate limits, or error handling for invalid inputs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences that efficiently cover the tool's purpose and key functionality. The first sentence front-loads the main action and process steps, while the second focuses on an important parameter nuance. No wasted words, though it could be slightly more structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 5-parameter mutation tool with no annotations and no output schema, the description provides good coverage of the action and process. However, it lacks information about return values, error conditions, or system constraints. Given the complexity of the operation (multiple side effects), more behavioral context would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 5 parameters thoroughly. The description adds some context about trade selection ('based on trade') and the overrideVendorPhone use case ('assign a specific vendor instead of the default'), but doesn't provide additional semantic meaning beyond what's in the schema descriptions. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('escalates a maintenance ticket to a specialist vendor') and distinguishes it from siblings like 'get_open_tickets' (read-only), 'notify_tenant' (notification only), and 'update_maintenance_status' (status update only). It explicitly mentions the multi-step process including vendor selection, status update, workflow triggering, and tenant notification.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('escalates a maintenance ticket to a specialist vendor') and mentions the overrideVendorPhone parameter for specific assignments. However, it doesn't explicitly state when NOT to use it or name alternatives among the sibling tools (e.g., when to use update_maintenance_status instead).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_open_ticketsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It effectively describes key behaviors: default filtering (non-resolved tickets), sorting logic (priority then recency), and that it's a retrieval operation. However, it doesn't mention potential limitations like pagination, rate limits, or authentication requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences that efficiently cover purpose, filtering parameters, defaults, and sorting logic with zero wasted words. The description is appropriately sized and front-loaded with the core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only query tool with comprehensive schema documentation but no output schema, the description provides good context about filtering defaults and sorting behavior. It could be more complete by describing the return format or result structure, but given the tool's relative simplicity and good parameter documentation, it's mostly adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value beyond the schema - it mentions filtering by the same parameters but doesn't provide additional semantic context. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Retrieves maintenance tickets'), resource ('from the property management system'), and scope ('Filters by status, priority, category, and property'). It distinguishes from siblings like 'escalate_to_vendor' or 'update_maintenance_status' by being a read-only query tool rather than a mutation tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through filtering parameters and default behavior, but doesn't explicitly state when to use this tool versus alternatives like 'update_maintenance_status' for modifying tickets or 'notify_tenant' for communication. No explicit when-not-to-use guidance or prerequisite information is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• notify_tenantA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does well by disclosing key behavioral traits: it specifies the five message templates, fallback to SMS, and side effects (logs to n8n and appends to ticket internal notes). However, it doesn't mention potential rate limits, authentication needs, or error handling details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in three sentences: the first states the core action and templates, the second explains parameter behavior, and the third covers fallback and side effects. Every sentence adds essential information with zero wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description provides strong context about behavior, parameters, and side effects. It could be more complete by detailing the exact format of logged messages or clarifying error responses, but it covers the essential operational aspects well given the complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, providing detailed parameter documentation. The description adds some value by explaining the purpose of customMessage ('overrides the built-in template') and listing the five template types, but doesn't significantly enhance the schema's already comprehensive parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Sends a WhatsApp message') and target resource ('to the tenant associated with a ticket'), distinguishing it from sibling tools like escalate_to_vendor or update_maintenance_status by focusing on tenant communication rather than vendor escalation or status updates.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool (sending notifications to tenants about ticket updates) and mentions fallback behavior (SMS if WhatsApp fails), but does not explicitly state when not to use it or directly compare it to alternatives like escalate_to_vendor for vendor-related communications.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_maintenance_statusA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: it's a mutation tool (implied by 'Updates'), records history notes, fires webhooks for automation sync, and enforces status transition rules. However, it doesn't mention potential side effects like error handling or permission requirements, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with core functionality in the first sentence, followed by essential details in a logical flow. Every sentence earns its place: the first explains the primary action, the second adds automation context, and the third provides critical transition rules—all without waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description does well by covering purpose, usage rules, and behavioral context like webhook firing. It lacks details on return values or error responses, but given the schema's completeness and the description's focus on transitions and automation, it's largely sufficient for agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value beyond the schema by implying parameter interactions (e.g., 'scheduledFor' is contextually linked to 'in_progress' status), but doesn't provide additional syntax or format details. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Updates the status', 'records a history note', 'Fires an n8n webhook') and resources ('maintenance ticket', 'automation system'). It distinguishes itself from siblings like 'get_open_tickets' (read-only) and 'notify_tenant' (communication-focused) by emphasizing status transitions and system synchronization.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance by detailing valid status transitions (e.g., 'open→in_progress|awaiting_tenant|escalated'), which tells the agent when to use this tool versus alternatives. It implicitly suggests not using it for invalid transitions, though it doesn't name specific sibling alternatives for edge cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.