Server Quality Checklist

Tool Scores

• create_eventA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=false, destructiveHint=false, idempotentHint=false, and openWorldHint=false, so the agent knows this is a non-destructive, non-idempotent write operation. The description adds useful context about the prerequisite (list_calendars) and what fields are created, but doesn't disclose additional behavioral traits like error conditions, rate limits, or authentication requirements beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences: the first states the purpose, the second provides prerequisite and parameter overview. Every sentence earns its place with essential information, and it's appropriately front-loaded with the core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a creation tool with no output schema, the description provides good context about prerequisites and parameters. However, it doesn't describe what happens on success (e.g., returns event ID) or failure scenarios. Given the 6 parameters and mutation nature, slightly more behavioral context would be helpful, but the prerequisite guidance is strong.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter clearly documented in the schema. The description adds marginal value by summarizing the parameters ('title, times, and optional description/location') and mentioning the calendarUrl prerequisite, but doesn't provide additional semantic context beyond what's already in the well-documented schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Create a new calendar event') and resource ('calendar event'), distinguishing it from siblings like delete_event, update_event, get_event_details, and list_events. It specifies the core functionality of creating events with title, times, and optional fields.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides a prerequisite ('You must first call list_calendars to get the calendarUrl') and distinguishes when to use this tool versus alternatives by naming the specific prerequisite tool. This gives clear guidance on the required sequence of operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_eventsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only, non-destructive, and idempotent behavior, but the description adds valuable context about the return format (iCalendar data with specific fields like url, etag, and data) and parsing instructions, which goes beyond what annotations provide without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Front-loaded with the core purpose and prerequisite, followed by return details and parsing instructions. The sentences are efficient, but the description could be slightly more streamlined by integrating some details (e.g., the parsing note might be condensed).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of an output schema, the description compensates by detailing the return structure (url, etag, data) and iCalendar format, which is essential for understanding the tool's output. However, it doesn't mention potential limitations like pagination or error cases, leaving minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents the three required parameters. The description adds minimal semantic context beyond the schema, such as linking calendarUrl to list_calendars output, but doesn't provide additional parameter details, meeting the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get events from a calendar') and resource ('events'), distinguishing it from siblings like create_event, delete_event, get_event_details, and update_event by focusing on listing/retrieval rather than modification or detailed single-event operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states a prerequisite ('You must first call list_calendars to get the calendarUrl') and provides clear context on when to use this tool (for retrieving events within a date range), with implicit alternatives being other event-related tools for different operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_event_detailsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate this is a read-only, non-destructive, idempotent operation with a closed world. The description adds valuable context beyond annotations by specifying the prerequisite calls needed and that it returns structured data instead of raw format, which helps the agent understand behavioral constraints not covered by annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by prerequisite information and output clarification. Every sentence adds value—none are redundant or verbose—making it efficiently structured and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only tool with good annotations and a simple parameter schema, the description is largely complete. It covers purpose, prerequisites, and output format. However, without an output schema, it could benefit from more detail on the structured data fields (e.g., explicitly listing title, start, end, location, description) to fully compensate for the missing output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the eventUrl parameter clearly documented as required and sourced from list_events output. The description reinforces this by mentioning the prerequisite to get the eventUrl from list_events, but doesn't add significant semantic detail beyond what the schema already provides. Baseline 3 is appropriate given high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get parsed details') and resource ('a specific event'), distinguishing it from siblings like list_events (which lists events) or create_event (which creates events). It explicitly mentions returning structured event data instead of raw iCalendar format, adding further differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit prerequisites ('You must first call list_calendars, then list_events to get the eventUrl') and clarifies when to use this tool versus alternatives by specifying it's for getting details of a specific event after identifying it through list_events. This gives clear operational context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_eventA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds valuable behavioral context beyond annotations: it specifies the prerequisite workflow (calling list_calendars and list_events first) and the partial-update behavior ('Only include fields you want to change; omitted fields stay the same'). Annotations cover read/write status (readOnlyHint=false), idempotency (idempotentHint=true), and non-destructiveness (destructiveHint=false), but the description complements this with practical usage details. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded: the first sentence states the core purpose, followed by prerequisite and behavioral details in two additional sentences. Every sentence earns its place by providing critical guidance without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (mutation with prerequisites), rich annotations (covering safety and idempotency), and full schema coverage, the description is nearly complete. It adds key contextual details like prerequisites and partial-update behavior. The main gap is lack of output schema or description of return values, but annotations help mitigate this. It's well-suited for the agent's needs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with all parameters well-documented in the input schema (e.g., eventUrl is REQUIRED with an example, others are optional with clear purposes). The description adds minimal parameter semantics beyond the schema, only reinforcing the eventUrl prerequisite and partial-update approach. This meets the baseline of 3 when schema coverage is high.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Modify an existing event') and resource ('event'), distinguishing it from siblings like create_event (creates new), delete_event (removes), get_event_details (reads), and list_events (lists). It goes beyond just restating the name/title by specifying it's for modifying existing events.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit prerequisites ('You must first call list_calendars, then list_events to get the eventUrl'), which tells the agent exactly when and how to prepare for using this tool. It also implicitly distinguishes from alternatives by focusing on modification rather than creation, deletion, or listing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_calendarsA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover safety (readOnlyHint: true, destructiveHint: false) and idempotency, but the description adds valuable behavioral context: that this returns an array with specific fields (displayName, url, timezone), that the URL is required for other operations, and practical interpretation guidance. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear step-by-step guidance, but slightly verbose with repetition of 'calendar' and could be more front-loaded. Every sentence adds value, but some phrasing could be tighter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with comprehensive annotations, the description provides complete context: clear purpose, explicit prerequisites, output structure explanation, and practical usage guidance. No output schema exists, so the description appropriately explains what's returned.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0 parameters and 100% schema description coverage, the baseline is 4. The description appropriately doesn't discuss parameters since none exist, focusing instead on output semantics and usage context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Lists all calendars') and resource ('in the user's Fastmail account'), and distinguishes it from siblings by explaining its prerequisite role for other calendar operations. It provides concrete examples of calendar types like 'Work' and 'Personal'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use this tool ('ALWAYS CALL THIS FIRST' before list_events, create_event, update_event, or delete_event) and why (to get the calendar URL). It also provides guidance on how to interpret results ('Look at the displayName to identify which calendar the user wants').

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_eventA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  While annotations already indicate destructiveHint=true and idempotentHint=true, the description adds crucial behavioral context: it explicitly states 'This cannot be undone' (reinforcing destructiveness), explains the purpose of the etag parameter ('prevents accidentally deleting a modified event'), and provides workflow prerequisites. This goes well beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured with three distinct, front-loaded components: the core action in all caps, prerequisites, and warnings. Every sentence earns its place by providing essential information without redundancy. It's brief yet comprehensive for a dangerous operation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool with no output schema, the description provides complete context: it explains the irreversible nature, prerequisites, confirmation requirement, and parameter sourcing. Given the high-stakes nature of deletion and the comprehensive annotations, this description leaves no critical gaps for the agent to understand when and how to use this tool safely.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the baseline is 3. The description adds value by explaining why both parameters are needed ('You must first call list_calendars, then list_events to get both the eventUrl AND etag') and providing context about their source, which helps the agent understand the required workflow beyond just parameter definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('PERMANENTLY DELETE') and resource ('an event'), distinguishing it from sibling tools like create_event, update_event, and get_event_details. It uses strong, unambiguous language that leaves no doubt about the tool's function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit prerequisites ('You must first call list_calendars, then list_events to get both the eventUrl AND etag') and clear warnings about when to use it ('Always confirm with the user before deleting'). It also implicitly distinguishes from alternatives like update_event by emphasizing the irreversible nature of deletion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.