Server Quality Checklist

Tool Scores

• search_assignmentsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the search action but doesn't describe key behaviors: whether results are paginated, sorted, or limited; if it requires authentication; potential rate limits; or what the output format looks like (e.g., list of assignments with details). For a search tool with zero annotation coverage, this leaves significant gaps in understanding how it operates.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Searches for assignments across all courses') and lists key filters. There's no wasted text, and it's appropriately sized for the tool's complexity. However, it could be slightly more structured by separating search criteria from scope for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (5 parameters, search functionality) and lack of annotations and output schema, the description is incomplete. It doesn't address behavioral aspects like result handling, authentication needs, or error cases, which are critical for an AI agent to use it correctly. The description alone is insufficient for safe and effective tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions searchable fields (title, description, due dates, course filters), which aligns with parameters like 'query', 'dueBefore', 'dueAfter', and 'courseId'. However, with 100% schema description coverage, the input schema already documents all 5 parameters thoroughly (e.g., date formats, defaults). The description adds minimal value beyond reinforcing the schema, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Searches for') and resource ('assignments across all courses'), specifying what the tool does. It mentions searchable fields (title, description, due dates, course filters), which helps distinguish it from siblings like 'get_assignment' (singular retrieval) and 'list_courses' (different resource). However, it doesn't explicitly differentiate from 'canvas_list_active_courses' in terms of scope or output type.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, compare with sibling tools (e.g., use 'get_assignment' for a specific assignment, 'list_courses' for course listing), or specify scenarios where this search is preferred. Usage is implied by the search functionality but lacks explicit context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_coursesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions filtering options but fails to cover critical aspects such as whether this is a read-only operation, potential rate limits, authentication requirements, or the format of returned data (e.g., pagination). This leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Lists all courses you are enrolled in') and adds necessary detail about filtering. It avoids redundancy and wastes no words, though it could be slightly more structured for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no output schema, no annotations), the description is adequate but incomplete. It covers the basic purpose and parameter context but lacks details on behavioral traits and output, which are needed for full agent understanding despite the simple schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already fully documents the single parameter 'state' with its enum values and default. The description adds minimal value by mentioning the filtering options, but doesn't provide additional semantics beyond what the schema specifies, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Lists') and resource ('courses you are enrolled in'), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'canvas_list_active_courses' or 'search_assignments', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through the phrase 'with options to filter by active, completed, or all courses,' suggesting when to use different parameter values. However, it lacks explicit guidance on when to choose this tool over alternatives like 'canvas_list_active_courses' or 'get_assignment,' leaving room for ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_assignmentA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool retrieves information (implying read-only behavior) and specifies the types of details returned. However, it doesn't mention potential errors (e.g., invalid IDs), authentication needs, rate limits, or response format, leaving gaps in behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose and lists key details without unnecessary words. Every part earns its place by specifying what is retrieved, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description provides basic context but is incomplete. It covers the purpose and output content but lacks details on error handling, authentication, or return structure. For a read tool with full schema coverage, this is minimally adequate but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description doesn't add any meaning beyond what the schema provides (e.g., it doesn't explain how 'formatType' affects the output or provide examples). Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('retrieves') and resource ('specific assignment') with specific details about what information is included ('description, submission requirements, and embedded links'). It distinguishes from the sibling 'search_assignments' by focusing on a single assignment rather than searching multiple, though it doesn't explicitly mention this distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving detailed information about a specific assignment, but it doesn't explicitly state when to use this tool versus alternatives like 'search_assignments' or provide any exclusions. The context is clear but lacks explicit guidance on tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• canvas_list_active_coursesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It adds useful context: it specifies the API used ('dashboard API') and a performance trait ('Much faster than list_courses'). However, it doesn't disclose other behavioral aspects such as authentication requirements, rate limits, pagination, or what 'active/current' means precisely (e.g., based on enrollment status or term dates). The description doesn't contradict any annotations, but it could be more comprehensive given the lack of annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise and well-structured: two sentences that efficiently convey the tool's purpose and key advantage. Every sentence adds value—the first defines what it does, and the second provides a performance comparison. It's front-loaded with the core functionality, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (simple listing with no parameters) and the lack of annotations and output schema, the description is somewhat complete but has gaps. It covers the purpose and a performance hint but doesn't explain what 'active/current' entails, the return format, or any error conditions. For a tool with no structured behavioral data, more context on these aspects would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so the schema fully documents that no inputs are required. The description doesn't need to add parameter details, but it implicitly confirms this by not mentioning any parameters. Since there are no parameters, the baseline is 4, as the description doesn't detract from or conflict with the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Lists only your active/current courses using the dashboard API.' It specifies the verb ('Lists'), resource ('active/current courses'), and method ('dashboard API'), which is clear and specific. However, it doesn't explicitly distinguish this tool from its sibling 'list_courses' beyond mentioning it's 'much faster'—it could more directly contrast their scopes (active vs. all courses).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides good usage guidance by stating when to use this tool: 'Lists only your active/current courses' and comparing it to an alternative: 'Much faster than list_courses.' This implies that 'list_courses' is a sibling tool for broader listing, but it doesn't explicitly state when not to use this tool (e.g., for archived or all courses) or name 'list_courses' as the direct alternative for non-active courses.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.