Server Quality Checklist

Tool Scores

• complete_taskC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions 'fuzzy match' behavior, which is useful context beyond the schema, but doesn't disclose other traits like whether completion is reversible, requires permissions, affects subtasks, or returns confirmation. For a mutation tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste—it directly states the purpose and key behavioral detail (fuzzy matching). It's appropriately sized and front-loaded, earning its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a mutation tool with no annotations and no output schema, the description is incomplete. It lacks details on behavioral outcomes (e.g., what 'complete' entails, error handling), and while it mentions fuzzy matching, it doesn't cover other critical aspects like return values or side effects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the 'task_name' parameter fully. The description adds value by specifying 'fuzzy match' for the name, which clarifies semantics beyond the schema's basic description. Baseline 3 is appropriate when schema does heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Mark as complete') and resource ('task'), specifying it's found by name with fuzzy matching. It distinguishes from siblings like 'create_task' or 'update_task' by focusing on completion, but doesn't explicitly differentiate from other potential completion-related tools (though none are listed).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'update_task' (which might also mark tasks complete) or other siblings. The description implies usage for completing tasks by name, but lacks explicit when/when-not instructions or prerequisite context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_subtasksC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action is 'Create multiple subtasks,' implying a write operation, but doesn't cover critical behaviors like error handling (e.g., if parent isn't found), side effects, permissions required, or response format. The fuzzy match detail adds some context, but overall, behavioral traits are under-specified for a mutation tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('Create multiple subtasks under a parent task') and adds necessary detail ('Parent is found by name (fuzzy match)'). Every word earns its place, with no redundancy or fluff, making it highly concise and well-structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity as a mutation operation with no annotations and no output schema, the description is incomplete. It lacks details on error scenarios, what happens on success (e.g., return values), or how subtasks are structured (e.g., ordering, IDs). While concise, it doesn't compensate for the missing structured data, leaving gaps for an agent to operate effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both parameters ('parent_task_name' and 'subtask_strings') documented in the schema. The description adds minimal value beyond the schema by mentioning 'fuzzy match' for the parent, which clarifies the matching behavior but doesn't provide additional syntax or format details. This meets the baseline of 3 for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create multiple subtasks') and the target resource ('under a parent task'), with the specific mechanism of parent identification ('found by name (fuzzy match)'). It distinguishes from siblings like 'create_task' by focusing on subtasks rather than standalone tasks, though it doesn't explicitly mention all siblings. The purpose is specific and actionable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'create_task' for standalone tasks or 'update_task' for modifying existing tasks. It mentions the parent is found by fuzzy match, which hints at usage context, but lacks explicit when/when-not instructions or prerequisites. This leaves the agent to infer usage from tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_taskC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states this is an update operation, implying mutation, but doesn't cover critical aspects like required permissions, whether changes are reversible, error handling (e.g., if task not found), or response format. The mention of 'found by name' hints at lookup behavior but lacks depth on how matching works or potential side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('Update a task found by name') and follows with specifics on modifiable fields. There's no wasted text, and it's appropriately sized for the tool's complexity. However, it could be slightly more structured by separating usage context from parameter details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (7 parameters, mutation tool), lack of annotations, and no output schema, the description is incomplete. It covers the basic purpose and parameters but misses behavioral context (e.g., permissions, errors), output details, and usage guidelines. For a tool with significant mutation potential and low schema coverage, this leaves gaps that could hinder correct agent invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is low at 29%, with only 'task_name' and 'due_string' having descriptions. The description adds value by listing updatable fields ('content, description, due date, priority, project, or section'), which maps to most parameters, but doesn't provide syntax details (e.g., format for 'due_string' beyond 'natural language date') or clarify optional vs. required beyond the schema. It partially compensates for the coverage gap but not fully.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Update a task') and resource ('found by name'), with specific mention of what can be changed ('content, description, due date, priority, project, or section'). It distinguishes from siblings like 'create_task' and 'complete_task' by focusing on updates rather than creation or completion. However, it doesn't explicitly differentiate from 'create_subtasks' which might also modify tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., task must exist), exclusions (e.g., cannot update completed tasks), or comparisons to siblings like 'complete_task' for marking tasks as done. Usage is implied through the action but lacks explicit context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_projectsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the output format (IDs, names, section names) but lacks details on permissions, rate limits, pagination, or whether it's a read-only operation. The phrase 'List all' implies completeness but doesn't clarify scope or limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with zero waste. The first sentence states the purpose and output, and the second provides usage guidance, making it efficiently front-loaded and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is adequate but has gaps. It explains what the tool does and hints at usage, but without annotations or output schema, it should more fully describe behavioral aspects like safety, response format, or limitations to be complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately doesn't discuss parameters, earning a baseline score of 4 for not adding unnecessary information beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('Todoist projects') with specific output fields (IDs, names, section names). It distinguishes from siblings like 'list_tasks' by focusing on projects rather than tasks, though it doesn't explicitly mention this distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage ('Use for routing context'), suggesting it's for obtaining project metadata to route other operations. However, it doesn't explicitly state when to use this versus alternatives like 'list_tasks' or provide any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tasksB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states this is a list operation with filtering, which implies read-only behavior, but doesn't disclose important traits like pagination, sorting, rate limits, authentication needs, or what happens when no filters are applied. The description provides basic behavioral context but misses critical operational details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('List tasks') followed by filter details. Every word earns its place with no redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description provides adequate basics for a list operation but lacks completeness. It covers parameters well but doesn't address return format, error conditions, or system constraints. For a tool with 4 parameters and no structured metadata, more context would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates well by explaining all 4 parameters and their semantics: project_name, section_name, priority (with range 1-4), and due_today. This adds significant value beyond the bare schema, though it doesn't provide format examples or deeper constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and resource 'tasks', and specifies optional filtering capabilities. It distinguishes from siblings like create_task or update_task by focusing on retrieval rather than mutation. However, it doesn't explicitly differentiate from list_projects which lists a different resource type.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through the mention of 'optional filters' but doesn't provide explicit guidance on when to use this tool versus alternatives like list_projects or complete_task. No exclusions or prerequisites are mentioned, leaving the agent to infer appropriate contexts.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• parse_meeting_notesA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses key behavioral traits: it parses text to extract specific elements, creates Todoist tasks automatically, infers due dates and priorities, and optionally routes tasks to a project. However, it lacks details on error handling, rate limits, authentication needs, or what happens if parsing fails, which are important for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with a single sentence that efficiently conveys the core functionality and an optional feature. Every word earns its place, avoiding redundancy and focusing on key actions and outcomes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (parsing and task creation), no annotations, and no output schema, the description is moderately complete. It covers the purpose and basic behavior but lacks details on output format, error cases, or integration specifics. For a tool with no structured safety or output information, it should do more to compensate, but it provides a clear starting point.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('raw_notes' and 'target_project_name'). The description adds marginal value by mentioning that parsing extracts action items, decisions, etc., from 'raw_notes' and that routing is optional to a 'target project by name', but it does not provide additional syntax or format details beyond what the schema provides. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific verb ('parse') and resource ('raw meeting note text'), and it details the extraction targets (action items, decisions, follow-ups, deadlines) and the creation of Todoist tasks with inferred due dates and priorities. It distinguishes from sibling tools like 'create_task' by focusing on parsing and automated task creation from notes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: when you have raw meeting notes to parse into structured tasks. It implies an alternative by mentioning the optional routing to a target project, but it does not explicitly state when not to use it or name specific alternatives among siblings (e.g., 'create_task' for manual task creation).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_taskA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions that the tool 'supports' various features (project/section by name, natural language due date, etc.), which implies flexibility but doesn't detail error handling, authentication requirements, rate limits, or what happens on success/failure. It adds some context about parameter resolution (e.g., 'resolved to ID' implied for project_name), but lacks comprehensive behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with the core purpose ('Create a single Todoist task') followed by a concise list of supported features. Every sentence adds value without redundancy, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (8 parameters, no annotations, no output schema), the description is adequate but has gaps. It covers the tool's purpose and key parameters well, but lacks details on behavioral aspects like error conditions, response format, or prerequisites. For a creation tool with moderate parameter count and no structured safety hints, more completeness would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context beyond the input schema by listing key parameters (project/section by name, natural language due date, priority 1-4, labels, optional parent task) and clarifying semantics like 'natural language' for due dates and 'subtask' for parent tasks. With schema description coverage at 63%, this compensates well for gaps, though it doesn't cover all 8 parameters explicitly (e.g., 'description' field is omitted in the description).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Create a single Todoist task') and resource ('Todoist task'), distinguishing it from siblings like 'update_task' or 'complete_task' which modify existing tasks rather than creating new ones. The verb 'Create' is precise and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool (creating a single task with various optional features), but doesn't explicitly mention when not to use it or name alternatives. For example, it doesn't contrast with 'create_subtasks' for batch creation or 'update_task' for modifications, though the purpose implies this distinction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.