Server Quality Checklist

Tool Scores

• calcom_list_appointmentsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the tool lists appointments and returns details, but doesn't disclose behavioral traits such as authentication requirements, rate limits, pagination, error handling, or what 'details' include. For a read operation with no annotation coverage, this leaves significant gaps in understanding how the tool behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with three sentences that are front-loaded: it states the purpose, filtering capability, and return value efficiently. However, the second sentence could be more integrated, and there's minor redundancy in mentioning 'lists' and 'returns', but overall it's well-structured with minimal waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity is low (a simple list operation with 2 parameters), schema coverage is high (100%), and no output schema is provided, the description is adequate but incomplete. It covers the basic purpose and filtering, but lacks details on return format, error cases, or behavioral context, making it minimally viable but with clear gaps for an agent to rely on.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents both parameters (startDate and endDate) with their formats. The description adds that it 'can be filtered by date range', which aligns with the schema but doesn't provide additional semantic context beyond what's in the schema. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Lists appointments from Cal.com calendar' with the verb 'lists' and resource 'appointments'. It distinguishes from siblings like 'add', 'delete', and 'update' by focusing on retrieval. However, it doesn't explicitly contrast with siblings beyond the verb difference, so it's not a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for listing appointments with date filtering, but doesn't explicitly state when to use this tool versus alternatives like 'calcom_add_appointment' or provide context on prerequisites. It mentions filtering by date range, which gives some guidance, but lacks explicit when/when-not instructions or named alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calcom_update_appointmentB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions 'Requires booking ID and the fields to update,' which hints at prerequisites but doesn't cover critical aspects like authentication needs, rate limits, error handling, or what happens on success/failure. For a mutation tool with zero annotation coverage, this leaves significant gaps in understanding the tool's behavior and constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, with two sentences that efficiently convey purpose and usage. Every sentence adds value: the first states what the tool does, and the second provides context and prerequisites. There's no redundant or wasted information, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a mutation operation with 4 parameters) and lack of annotations and output schema, the description is incomplete. It doesn't explain return values, error conditions, or behavioral details like whether updates are partial or full. For a tool that modifies data, more context is needed to ensure safe and correct usage by an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all parameters (bookingId, startTime, endTime, notes). The description adds minimal value beyond the schema by mentioning 'Requires booking ID and the fields to update,' which reinforces the required parameter but doesn't provide additional syntax or format details. Baseline 3 is appropriate when the schema handles most of the parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Updates an existing appointment in Cal.com calendar.' It specifies the verb ('Updates') and resource ('appointment in Cal.com calendar'), making the intent unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'calcom_add_appointment' or 'calcom_delete_appointment' beyond mentioning 'existing appointment,' which is implied but not directly contrasted.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: 'Use this for rescheduling or modifying existing appointments.' This gives practical scenarios (rescheduling, modifying) and implies it's for updates rather than creation or deletion. It doesn't explicitly state when not to use it or name alternatives like 'calcom_add_appointment' for new appointments, but the context is sufficient for basic guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calcom_add_appointmentA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While it mentions the tool creates appointments and lists required parameters, it doesn't disclose important behavioral traits like authentication requirements, error handling, rate limits, or what happens on success (e.g., returns appointment ID). This leaves significant gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences that efficiently convey purpose and parameters. It's front-loaded with the main action and could be slightly more structured but contains no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is incomplete. It doesn't explain what happens after creation (success response, error cases), authentication requirements, or system constraints. Given the complexity of creating calendar appointments, more contextual information is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, providing complete parameter documentation. The description adds minimal value beyond the schema by listing the parameters but doesn't provide additional context about their meaning, relationships, or usage patterns. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Creates a new appointment'), resource ('in Cal.com calendar'), and purpose ('for scheduling new meetings or appointments'). It distinguishes from sibling tools like delete, list, and update by focusing exclusively on creation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('for scheduling new meetings or appointments'), but does not explicitly mention when not to use it or name alternatives like the sibling tools. This gives good guidance but lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calcom_delete_appointmentA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool deletes appointments and requires a booking ID, but does not disclose critical behavioral traits such as permissions needed, whether deletions are reversible, rate limits, or what happens to associated data (e.g., notifications). For a destructive mutation tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with two concise sentences that directly address the tool's purpose and key requirement. Every sentence earns its place, with no wasted words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a destructive mutation tool with no annotations and no output schema, the description is incomplete. It lacks information on behavioral aspects (e.g., permissions, reversibility), does not explain return values or errors, and does not fully address the tool's complexity in a calendar management context. The description should do more to compensate for the missing structured data.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters (bookingId and reason) fully. The description adds minimal value beyond the schema by mentioning 'Requires booking ID' and implying the reason is for cancellation, but does not provide additional syntax, format, or usage details. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Deletes') and resource ('an existing appointment from Cal.com calendar'), and distinguishes it from siblings by focusing on deletion/cancellation rather than creation, listing, or updating. The phrase 'for canceling appointments' reinforces the purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('for canceling appointments'), but does not explicitly mention when not to use it or name alternatives (e.g., using calcom_update_appointment for modifications instead). It implies usage by stating the purpose, but lacks explicit exclusions or sibling comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.