Server Quality Checklist

Tool Scores

• get_eventB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behaviors. It implies a read operation ('get'), but does not explicitly confirm idempotency, side effects, authentication requirements, or rate limits. Since it's a simple retrieval, a score of 2 is given for minimal disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: a one-line summary followed by an Args section with clear parameter definitions. Every sentence is valuable, and the structure front-loads the purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists, the description does not need to explain return values. It covers purpose and parameters but lacks usage guidelines and behavioral transparency. For a simple tool, it is minimally complete but could be improved with contrasts to siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides examples for both parameters (event_id: 'evt-abc123', url: 'https://lu.ma/myevent' or 'myevent'), which adds meaning beyond the schema's generic 'anyOf string or null'. However, it does not clarify that at least one parameter should be provided, nor the behavior if both are given.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get full details for a single Luma event', specifying a specific verb and resource. It distinguishes from siblings like 'search_events' which lists multiple events, and 'export_event_ics' which is for exporting to ICS format.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It does not explain that this tool is for retrieving a single event by ID or URL, while search_events is for discovering events. Missing context on parameter choice or fallback behavior.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• export_event_icsA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It only states it generates an ICS string but does not mention idempotency, authentication requirements, error behavior (e.g., if event not found), or output format details beyond the name.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two short sections. Every sentence adds value: the main sentence states what it does, and the args section defines the two parameters. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with two optional parameters and an output schema, the description covers the essential purpose and parameter hints. It is mostly complete, though could note that at least one parameter is typically needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, meaning no descriptions in schema. The description adds minimal semantics: event_id is a Luma API ID with example, url is a URL or ID. However, it does not clarify that at least one is needed or how they relate, and both are optional with default null.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool generates an ICS calendar string for a Luma event, using the verb 'Generate' and specifying the resource. It distinguishes from siblings like get_event (retrieves details) and search_events (searches).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for adding to calendar but does not explicitly state when to use this tool versus alternatives or when not to use it. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_preferencesA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations present, so description carries full burden. It discloses persistence, response messages with agent instructions, and permanent effect of skip parameters. Could mention auth requirements or lack thereof.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Front-loaded with one-line purpose, then structured 'IMPORTANT for agents' section with bullet points. No wasted words; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given output schema exists (not shown), description adds context about response messages. All 5 parameters explained, no required params, handles edge cases like permanent decline. Complete for a preferences setter.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%; description adds full meaning for all 5 parameters: categories (exact slugs with mapping examples), address, max_distance_miles, skip_categories, skip_address (both with permanent decline explanation).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Save default search preferences (persists across restarts)' with a specific verb and resource. This distinguishes from siblings which are all event-related tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides important agent instructions including exact category slugs, mapping examples, and how to handle response messages and skip parameters. Lacks explicit when-not-to-use, but context is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_eventsA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description discloses key behaviors: two modes, default window, login process, first-run prompting, and the presence of agent-facing instructions in the response. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-organized with clear sections (overview, modes, defaults, agent notes, args). Front-loaded with essential info. Some minor redundancy in arg descriptions (e.g., days overriding default), but overall efficient for 14 parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers all 14 parameters, both modes, and output format (messages array). Lacks explicit error handling or edge cases, but given the output schema is noted as existing, the description is sufficiently complete for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% property descriptions; the description fully compensates by explaining each parameter (city, category, keywords, etc.) with usage notes, accepted values, and default behaviors. Adds significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'Search for Luma events' and explains two distinct modes (home and travel) with concrete examples. Distinguished from sibling tools (export_event_ics, get_event, set_preferences) which handle export, single event retrieval, and preferences.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to use each mode, how to choose between category and keywords, and when to use days vs after/before. Also gives agent-specific instructions for handling messages in the response.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.