Server Quality Checklist

Tool Scores

• canvas_list_assignmentsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only says 'List assignments' and does not mention pagination, sorting, authentication, read-only guarantees, or output format. This is not misleading, but it adds almost no behavioral context beyond the tool's name.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one short sentence with no filler, and the key scope is front-loaded. While it is minimal, the length is appropriate for a simple listing tool and every word contributes to meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is a simple list operation with one required parameter, so the description covers the core call adequately. However, with no annotations and no output schema, it omits return-format details and any edge-case behavior, leaving some contextual gaps for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. The phrase 'for one Canvas course' clarifies that course_id identifies a single course, adding a bit of meaning beyond the raw schema. However, it does not name the parameter or explain its expected format or constraints beyond what the schema already shows.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the verb 'List', the resource 'assignments', and the scope 'one Canvas course'. This clearly distinguishes it from siblings like canvas_list_courses, which list courses. It does not explicitly reference the alternatives, so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus canvas_list_courses or canvas_get_upcoming. The only hint is the phrase 'List assignments', which implies its use but does not provide any exclusions, prerequisites, or alternative routing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• canvas_get_upcomingA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. 'Get' clearly signals a read operation and 'authenticated user' provides context, but the description does not disclose return format, pagination, or whether any special permissions beyond authentication are required. It is not misleading, but it is not deeply transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence with no filler. The core purpose, scoping, and user context are front-loaded and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the zero-parameter schema and low complexity, the description is largely complete for invoking the tool correctly. It states what will be returned (upcoming events and assignment deadlines) and for whom, though it could specify the exact meaning of 'upcoming' and the response shape more precisely.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is no parameter semantics burden. The schema is trivially complete, and the description does not need to explain inputs. The baseline of 4 applies here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('get') and a specific resource ('upcoming events and assignment deadlines'), scoped to the authenticated user. It is distinguishable from siblings like canvas_list_assignments through the 'upcoming' and 'deadline' focus, though it does not explicitly contrast itself with those alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'upcoming' qualifier implies this tool is for time-sensitive events and deadlines rather than general listings, but the description gives no explicit guidance on when to choose this over canvas_list_courses or canvas_list_assignments. Usage context is implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• canvas_list_coursesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full disclosure burden. It does add two meaningful behavioral facts: the result is scoped to the authenticated user, and only active courses are returned, implying completed or archived ones are filtered out. However, it is silent on ordering, pagination, and the precise meaning of 'active', leaving gaps for a tool that must stand on its description alone.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence with zero wasted words. The verb is front-loaded, and each qualifier ('authenticated user's', 'active') earns its place by adding scoping or filtering information rather than padding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only list tool with no output schema, the description covers the essential facts an agent needs: the action, the owner of the data, and the active-only filter. Minor omissions — defining 'active' and describing ordering or pagination — are low-stakes given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and an empty input schema, so schema coverage is trivially 100%. Per the rubric, 0 params warrants a baseline of 4; there is nothing for the description to explain about argument semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('List') and a precise resource ('the authenticated user's active Canvas courses'). The resource noun 'courses' clearly differentiates it from the siblings canvas_get_upcoming and canvas_list_assignments, which target different Canvas entities, so an agent can tell them apart without opening the schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use is implied by the resource naming — use this when the agent needs the user's roster of courses — but there is no explicit when-to-use guidance, no exclusions, and no mention of when to prefer canvas_list_assignments or canvas_get_upcoming instead. The routing decision is left entirely to inference from the sibling names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.