Server Quality Checklist

Tool Scores

• create_eventA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and the description only says 'create event using natural language'. It gives no information about permissions, side effects, error conditions, or outcome beyond creation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with the verb and resource, no unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description does not mention what the tool returns or how optional parameters (calendar, notes) affect behavior. The example covers only the required parameter.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so description must compensate. It explains the 'sentence' parameter via example but offers no detail about 'notes' or 'calendar'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Create event' as the action and resource, and distinguishes from sibling tools that only read or delete events. The example 'Meeting tomorrow at 3pm' reinforces the natural language aspect.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Describes use case ('using natural language') with a concrete example, but does not explicitly mention when not to use the tool or suggest alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_eventsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses key behaviors: returns compact tab-separated output (id, title, startDate, endDate, attendeeCount), events are cached, and accepts natural language date strings. No annotations provided, so description carries full burden and does so well, though rate limits or auth are not mentioned.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Efficient and well-structured: 4 sentences covering purpose, date syntax, usage strategy, and output format. No redundant information; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While output schema exists to document return structure, the description omits guidance on the 'calendar' parameter, which is critical for filtering. Caching behavior is noted but lacks details like duration. This gap leaves agents potentially misusing the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Adds meaning for date parameters by specifying accepted formats ('today', 'yesterday', YYYY-MM-DD) and default behavior (today only). However, the 'calendar' parameter is completely undocumented in the description, and the range when to_date is null is ambiguous, leaving gaps despite 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool's purpose: 'List calendar events in a date range.' It uses specific verbs and distinguishes itself from siblings by advising to use get_event_details for full data, indicating this is a compact listing tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Offers concrete usage guidance: start with 2-week chunks, expand if sparse, and mentions it can be called in parallel. Also references get_event_details for full data, but does not explicitly contrast with search_events or other siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_calendarsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description provides key behavioral details: it is fast, uses JXA directly, returns plain text with one calendar per line as 'name (id)'. This sufficiently discloses behavior for a read-only list operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences, each serving a distinct purpose: what it does, performance/implementation note, output format. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and an output schema, the description adequately explains the output format. It does not cover error handling, but for a simple list tool it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, and schema coverage is 100%. The baseline for no parameters is 4. The description does not need to add parameter info but does explain the output format, which adds value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List all Fantastical calendars.' It is a specific verb-resource pair that distinguishes from sibling tools like list_events which lists events, not calendars.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies it is the go-to tool for listing calendars but does not explicitly state when to use it vs alternatives. It mentions speed and direct JXA usage, but no clear when-not or exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_eventsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses default date window, output format (tab-separated with caching), and support for parallel calls. No destructive behavior is mentioned, but it appropriately implies read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with four sentences, each serving a clear purpose: stating the action, default window, usage strategy, and output characteristics. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, explanation of return values is not needed. The description covers key aspects: search by title, date range control, chunking advice, and output format. Minor omission of error handling or limits prevents a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must compensate. It explains query (by title), and from_date/to_date (override, YYYY-MM-DD format). It adds meaning beyond schema names, though it does not explicitly mention default null behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches events by title, which is a specific verb+resource. It distinguishes from sibling list_events by mentioning the same output format but search functionality, and indicates default window and parallel calling capability.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on default window (±30 days), suggests starting with 2-week chunks and expanding, and allows parallel calls. However, it does not explicitly contrast with list_events or other siblings for when to use which.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• clear_cacheA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses that it clears caches and specifies which caches (event and attendee). No contradictions or omissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first states action, second gives usage. No extraneous words, highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no params, straightforward action), the description fully covers what it does and when to use. Output schema exists but is not needed to explain behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so baseline is 4. The description does not add parameter info, but none needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it clears in-memory event and attendee caches. Distinguishes from sibling tools that are CRUD/list operations, making its utility purpose explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use when starting a fresh analysis session,' providing clear context. However, no mention of when not to use or alternative tools, though none exist among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_event_detailsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses caching behavior ('Events are cached from previous...calls') and lazy loading of attendees ('Attendees are fetched lazily on first request and cached for the session'). It also lists returned fields. It does not mention rate limits or permissions, but for a read-only getter that is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each serving a distinct purpose: stating the core function, listing return fields, and explaining caching behavior. There is no redundant information, and the purpose is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given complexity (1 param, no nested objects, has output schema), the description covers the essential aspects: what it gets, what fields are returned, and how data is sourced (cached from previous calls, lazy loading). This is sufficient for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter is 'event_id', which is a required string. The schema coverage is 0%, so the description must add meaning. It does so by clarifying the ID must come from a cached event ('cached event by ID'), implying it is obtained from previous list/search calls. This adds context beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a clear verb+resource pair: 'Get full details of a cached event by ID.' It explicitly states the action (get) and the specific entity (cached event details by ID). This distinguishes it from sibling tools like list_events, list_calendars, and search_events, which have different scopes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that 'Events are cached from previous list_events/search_events calls,' providing clear context on when to use this tool (after listing/searching). It implies the event ID must come from those calls. However, it does not explicitly state when not to use it or mention alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.