Server Quality Checklist

Tool Scores

• list_assignmentsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already state readOnlyHint=true and destructiveHint=false. The description adds that the results include due dates, points, and submission status, plus bucket filtering. However, it does not disclose behaviors like pagination, unpublished assignments, or rate limits, so value beyond annotations is moderate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first clearly states the tool's purpose, and the second efficiently explains bucket usage. No wasted words, front-loaded with the main action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With strong annotations and an existing output schema, the description covers the core purpose and filter options. The main gap is not pointing to related tools like whats_due or get_assignment, which would help an agent choose correctly in ambiguous cases. Overall, it's nearly complete for a read-only list tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameters are well-documented. The description adds meaning for bucket by describing its filtering purpose, but only lists a subset of allowed values (upcoming, overdue, unsubmitted, past) versus the full enum (which also includes undated, ungraded, future). This is a minor enrichment over the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists a course's assignments with due dates, points, and submission status. This specific verb+resource+detail distinguishes it from siblings like get_assignment (single assignment), whats_due (due items), and my_submission (individual submission).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says to use bucket for filtering, but does not explicitly mention when to use this tool versus alternatives like whats_due or get_assignment. The context is implied by the purpose, but no exclusions or alternatives are named.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• my_submissionA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, so the agent knows this is a safe read operation. The description adds the scoping constraint 'user's own' and lists the specific data fields, but does not disclose behavior for missing submissions or authentication requirements. This is consistent with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single 18-word sentence that front-loads the verb and object, then lists the return contents. Every word adds value; there is no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 required params), rich annotations, and presence of an output schema, the description is adequate. It clearly states the tool's purpose and output domains, though it omits explicit usage guidance and edge-case behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with both parameters described as 'Course id' and 'Assignment id'. The description does not add any parameter-specific details beyond the schema, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Get' and identifies the resource as 'the user's own submission for an assignment', clearly distinguishing it from sibling tools like get_assignment or my_grades. It also enumerates the returned fields (state, score, grade, lateness, instructor comments, rubric assessment), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when the user needs their own submission details for a specific assignment, but it does not explicitly contrast with my_grades or get_assignment. No exclusions or alternative tool references are provided, though the 'user's own' scope gives clear context for when this tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_assignmentA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is clear. The description adds value by disclosing the exact scope of the response (instructions, dates, points, etc.), which goes beyond the annotations and helps the agent understand what 'full' means.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, information-dense sentence that front-loads the purpose ('Get one assignment in full') and lists the key contents without any filler. Every word contributes to understanding the tool's behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (two simple parameters) and the presence of an output schema that describes the return structure, the description is complete. It clearly states what the tool returns, and annotations cover the safety and idempotency aspects, leaving no significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage for both parameters (course_id and assignment_id) with simple descriptions. The tool description does not add any extra meaning or usage context for the parameters, so it earns the baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') and clearly identifies the resource ('one assignment in full') with an enumerated list of contents (instructions, due/lock dates, points, submission types, rubric, user's submission state). This distinguishes it from sibling tools like list_assignments, which lists assignments rather than fetching a single one in detail.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for retrieving a single assignment's full details when you have a specific course_id and assignment_id, but it does not explicitly state when to use this over alternatives like list_assignments or my_submission. No exclusions or alternative tool names are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• my_gradesA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safe-read nature is fully covered. The description adds the nuance that grades are 'current' and that results can span all courses or one, but doesn't reveal further behavioral details, which is acceptable given the annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with front-loaded purpose and concise usage guidance. Every word earns its place, with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With one optional parameter, full schema coverage, an output schema present, and strong annotations, the description covers both selection and invocation needs. No important context is missing for this simple read-only tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers course_id 100% with 'Limit to one course; omit for all,' and the description echoes this behavior without adding extra syntax or format details. A baseline of 3 is appropriate when the schema already documents the parameter fully.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Report' and clearly states it returns the user's current grade and score across all courses or a single course when course_id is given. This differentiates it from sibling tools like my_courses or whats_due by focusing on grade standing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Use this for "how am I doing" and standing questions,' providing clear when-to-use context. It doesn't mention exclusions or alternative tools, so it stops short of a full decision tree.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fileA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true and destructiveHint=false, so the safety profile is known. The description adds useful behavioral context about supported formats (PDF, DOCX, PPTX, plain text) and truncation behavior ('Long files are truncated to max_chars'), which goes beyond what annotations state and helps set expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three succinct sentences deliver the core purpose, supported formats, source of ids, and truncation limit. Every sentence earns its place, and the description is front-loaded with the primary action and outcome.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a straightforward read tool with strong annotations and an existing output schema, the description covers the essential operational details: file types, id sourcing, and truncation. Minor gaps include error handling or exact return format, but these are adequately compensated by the schema and annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both file_id and max_chars already documented. The description reinforces the source of file_id (from list_files) and the role of max_chars in truncation, but does not significantly enhance the schema's clarity. Baseline 3 is appropriate when the schema already carries the semantic load.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Download') and clearly states the resource ('Canvas file') and the outcome ('return its text'). It lists example content types (lecture slides, notes, readings) and distinguishes itself from siblings like list_files and get_page by focusing on file content extraction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly indicates a prerequisite: get file ids from list_files or course_content, which tells the agent when to use this tool. It does not explicitly mention alternatives or exclusions, but the context is specific enough for effective selection among sibling reading tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• course_announcementsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With readOnlyHint=true and destructiveHint=false, the safety profile is known. The description adds behavioral context by specifying 'active courses' and 'recent' (time window), and clarifies the filtering behavior when course_id is provided. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The entire description is one concise sentence that front-loads the verb 'List' and immediately states the resource and scope. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has low complexity, a clear output schema (per context signals), and annotations covering safety. The description accounts for the two parameters and the default scope, making it sufficiently complete for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters are fully described in the schema (100% coverage). The description reiterates the course_id filter behavior but does not add new semantic details beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' with the resource 'course announcements' and clearly states the scope ('across all active courses, or one course if course_id is given'). This distinguishes it from sibling tools like read_discussion and course_content by naming the announcement resource.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool: to list announcements, optionally filtered to a single course. It doesn't explicitly name alternatives or exclusions, but the scope statement ('across all active courses') implies usage coverage. The difference from siblings is implicitly clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_pageA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, covering the safety profile. The description adds context about the content type and parameter source, but doesn't disclose additional behavioral traits such as auth requirements or pagination. This is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no wasted words. The key action and resource are front-loaded, and the additional guidance about the syllabus is a useful comparison.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and safe, with strong annotations and an output schema present. The description covers its purpose, usage context, and a key parameter source, making it complete for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents both parameters. The description adds value by explaining that page_url is the page slug and is available from course_content, which helps agents locate the correct value beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets the content of a Canvas wiki page, using a specific verb and resource. It also distinguishes itself from get_syllabus, making it easy to understand its unique purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context (weekly overviews) and explicitly directs users to get_syllabus for the course syllabus. It also notes that page_url is available from course_content, which helps with usage. However, it doesn't explicitly list exclusions for other sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_filesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover read-only, idempotent, non-destructive behavior, lowering the bar. The description adds useful context about return fields (name, type, size) and the optional search filter, which helps the agent understand what to expect from a call. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary purpose, then the optional filter, then a clear pointer to read_file. Every sentence earns its place with zero redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With strong annotations, a complete output schema, and a simple listing use case, the description covers purpose, filtering, and the natural follow-up action (reading a file). No significant gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the description adds only a small restatement of the search filter (already in schema). It doesn't provide additional detail like input formats or edge cases beyond what the schema offers, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists files in a course with specific metadata (name, type, size), and distinguishes itself from read_file by pointing to that tool for text content. This goes beyond a simple restatement and effectively differentiates within the sibling set.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context that this is for listing files and explains how to filter with 'search'. It also names read_file as an alternative for getting the text of a single file. However, it doesn't explicitly exclude sibling tools like course_content or get_page, so it's not a full when/when-not guide.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• my_coursesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only, idempotent, and non-destructive behavior. The description adds context by specifying the exact data returned (course code, term, role) and its role in resolving course IDs for other tools. It does not contradict the annotations and provides useful scoping beyond the structured metadata.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences that are both information-dense and purposeful. The first sentence states the action and output; the second provides the key usage context. There is no redundancy, filler, or unnecessary detail, making it highly concise for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one optional parameter), full schema coverage, provided annotations, and an output schema, the description fully covers the necessary context. It explains what the tool does, what is returned, and the practical use case (resolving course_id), leaving no critical gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% because the single 'state' parameter has a clear description. The tool description does not add further parameter-level detail, but the schema already fully documents the parameter. Baseline of 3 is appropriate since the description does not need to compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and resource ('user's Canvas courses'), defines the returned fields (course code, term, role), and clearly distinguishes from sibling tools by stating its role in resolving course name/code to course_id. This is not a tautology and fully differentiates the tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: 'Use this to resolve a course name or code to the course_id that other tools require.' This gives clear context and a primary use case. However, it does not mention when not to use or explicitly name alternative tools, so it falls short of the highest bar.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• whats_dueA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, covering the safety profile. The description adds useful behavioral context: it spans all courses, includes multiple item types, and sorts soonest first. No contradictions with annotations, and the added scope/sort details are valuable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly two sentences, both information-dense and free of filler. It immediately states what the tool does and then gives use-case examples. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple listing tool with a single well-documented parameter and an output schema, so the description needn't explain return values. It covers purpose, scope, sorting, and primary use cases, making it complete for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'days' is fully described in the schema as 'Horizon in days to describe in the result', so baseline is 3. The description does not elaborate on the parameter beyond implying time horizon via 'what's due this week', but it doesn't need to since the schema covers it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List what is due for the user across all courses' with specific item types (assignments, quizzes, scheduled events) and sorting order. This distinguishes it from siblings like list_assignments and get_syllabus, making the tool's purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description calls it 'the primary tool for deadline planning' and gives example queries ('what's due this week', 'what do I have coming up'), which clearly signals when to use it. However, it does not explicitly mention alternatives or when not to use it, leaving some room for ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• canvas_requestA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (which already indicate destructive and non-read-only behavior), the description adds substantial context: irreversible changes for non-GET methods, permission model determined by Canvas account role, and the security implication that returned data is untrusted. This is rich, non-redundant disclosure that exceeds the annotation baseline.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is reasonably concise and front-loaded with the core purpose, followed by essential safety warnings. Each sentence earns its place, though some phrasing (e.g., 'What this is permitted to do is decided by Canvas...') is slightly verbose. It is not bloated but could be tightened.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity and dangerous nature, the description covers purpose, endpoint discovery, safety mechanisms (dry_run), irreversible side effects, permission model, and response handling. The presence of an output schema reduces the need to describe return values, and the provided guidance is sufficient for an agent to use the tool safely in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already covers all five parameters with descriptions, so the baseline is 3. The description adds marginal value by mentioning dry_run for previewing and noting that body is for write methods, but these are largely inferable from the schema. No significant additional semantic information is provided beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Executes any Canvas API endpoint directly' in the first sentence, establishing it as a direct, general-purpose API access tool. It distinguishes from siblings by emphasizing raw, uncurated behavior and explicitly references 'search_canvas_api' for discovery.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance to find endpoints with search_canvas_api first, and warns about the destructive nature of non-GET methods. It also suggests using dry_run to preview requests. While it doesn't exhaustively enumerate when to prefer curated tools over this one, it implicitly contrasts with 'curated tools' and provides a clear alternative for endpoint discovery.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• course_contentA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, and non-destructive behavior, so safety is covered. The description adds value by explaining the return structure (modules and item types) and the 'before fetching' workflow. It doesn't address openWorldHint implications (e.g., potential incomplete listings) but doesn't contradict annotations either.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, immediately stating the tool's purpose and usage. Every word earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple listing tool with one well-documented parameter, an output schema, and comprehensive annotations, the description sufficiently covers purpose, usage, and what to expect. It's complete for the agent to decide when and how to use it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully describes the single parameter (course_id, from my_courses), so baseline is 3. The description doesn't add parameter-specific detail beyond implying the input is a course, which is sufficient given 100% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool maps a course's structure—modules in order and items inside—using the specific verb 'map.' It distinguishes itself from siblings like list_files and get_page by emphasizing it's for discovering material before fetching it, not fetching content itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this to find what material exists before fetching any of it,' which clearly tells the agent when to use this tool (discovery phase) and implies alternatives (fetching tools) for actual content retrieval. It provides a clear context and usage directive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_syllabusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, so the description doesn't need to restate that. It adds valuable context by mentioning the return of course name and syllabus HTML, and explains the underlying reason for the tool's existence (Canvas stores syllabus on the course, not wiki page). Slight shortfall: no mention of error cases or authorization, but the read-only, idempotent nature is covered by annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero fluff. The first sentence states the core purpose and return value, the second gives the alternative rationale. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with one well-documented parameter and an output schema present, the description is complete. It explains what the tool returns, why it should be used over a sibling, and annotations cover the read-only/idempotent safety profile. No significant gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers the only parameter 'course_id' with a description 'Course id' (100% coverage), so the baseline is 3. The description adds no additional parameter-specific detail beyond what the schema provides. However, since there is only one parameter and it is fully documented, this is adequate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets a course's syllabus directly from Canvas, specifying the resource and what it returns (course name and syllabus HTML). It explicitly distinguishes itself from the sibling get_page by positioning this as the correct tool for syllabus retrieval, so the purpose is unambiguous and well differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Use this instead of get_page because Canvas stores the syllabus on the course, not as a wiki page.' This tells the agent exactly when to choose this tool and why, nailing the when-to-use and alternatives aspect.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_discussionA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, so the description adds context beyond that by explaining the flattened reply structure with a depth field. This is valuable behavioral detail not captured in the schema or annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with the main purpose and no redundant wording. Every clause adds information about behavior or parameter effects.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists and annotations are complete, the description is sufficient. It explains both usage modes and the nesting behavior, making it complete for a read-only tool with two parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameters are fully described, but the description adds meaning by explaining the behavior difference: omitting topic_id lists topics, while providing it returns the topic and replies. This goes beyond the schema's description of topic_id as 'omit to list topics' by clarifying the full output.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Read course discussions.' It distinguishes two modes: listing topics with only course_id, and retrieving a specific topic with replies using topic_id. This specificity differentiates it from sibling tools like post_discussion_reply, which is for writing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage context: 'With only course_id, lists the discussion topics. With topic_id, returns that topic and all its replies...' This tells the agent when to use each parameter. It does not explicitly name alternatives or exclusions, but the read vs. write distinction with sibling post_discussion_reply is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_canvas_apiA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the agent knows it's a safe read operation. The description adds the concrete return payload ('method, path, summary, and parameter names'), which provides useful behavioral context beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long and front-loads the core function ('Search all Canvas API endpoints by keyword'). Every clause earns its place, including the usage rule and the return format, with no redundant filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only search tool with a clear purpose, 3 well-documented parameters, and an output schema (as indicated by context signals), the description fully covers what an agent needs: what it does, when to use it, how to invoke it, and what to expect in return.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with clear descriptions for query ('Keywords, e.g. 'group membership''), limit ('Maximum results to return'), and method ('Optional filter: GET, POST, PUT, PATCH, DELETE'). The description does not add parameter-level detail beyond what the schema already provides, which is acceptable given the high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Search' and the resource 'all Canvas API endpoints', making the tool's function unmistakable. It also distinguishes itself from sibling tools by specifying it covers 'anything the curated tools do not cover', which sets clear boundaries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this to find the right endpoint for anything the curated tools do not cover', providing a clear when-to-use instruction. It also links to the next step ('then execute it with canvas_request'), offering a concrete workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• whoamiA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare read-only and idempotent behavior. The description adds specifics about what it returns (name, roles) and that it identifies the server's authentication, going beyond the structured annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence front-loaded with the action, followed by usage advice. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no parameters, comprehensive annotations, and an output schema, the description fully covers purpose and usage. It is simple and complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is trivially 100%. The baseline for no parameters is 4; the description doesn't need to add parameter details because none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool identifies the authenticated Canvas account and returns the user's name and role per course. It distinguishes from sibling tools like my_courses by focusing on identity/access rather than listing content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives explicit context: 'Call this first when you need to know what the user can access.' It provides clear when-to-use guidance, but does not explicitly mention alternatives or when-not-to-use scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_calendar_feed_urlA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnly, openWorld, idempotent), the description discloses critical security behavior: the URL is a bearer credential, grants full calendar access without authentication, and survives token rotation. It also advises caution about repeating the URL, adding real behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with the core purpose, followed by a security warning and usage constraints. Every sentence serves a purpose with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with an output schema, the description is complete: it explains what is returned (the URL), its security implications, and when to use it. The output schema likely describes the URL format, while the description covers the important caveats.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the baseline is 4. The description adds no parameter details because there are none; the schema is fully trivially covered. It doesn't need to compensate for any gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Fetch') and resource ('user's private calendar .ics subscription URL'), and explains its purpose as a subscription link for Google/Apple/Outlook to see Canvas deadlines. It clearly distinguishes from siblings like get_syllabus or course_content by focusing on the calendar feed URL.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to call (only when user explicitly asks for calendar feed/subscription link) and when not to (not during general orientation), plus not to repeat the URL unless asked. This provides clear usage boundaries without needing alternative tool mentions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• post_discussion_replyA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the destructiveHint annotation, the description adds key behavioral traits: the reply is public, under the user's own name, visible immediately to the whole class and instructor, and cannot be deleted. The dry-run behavior and data-vs-user warning are also disclosed, with no contradiction to annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, each delivering essential safety and functionality information without redundancy. The description is front-loaded with the tool's purpose and immediately proceeds to critical usage constraints, making it efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, visibility, irreversibility, confirmation requirement, dry-run usage, and a subtle prompt-injection caveat. For a destructive write tool with an output schema, this provides comprehensive context for safe invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so parameters are already well-documented. The description adds meaningful context for dry_run (safety preview) and message (exact text requiring confirmation), enriching the parameter semantics beyond the schema baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Posts a reply') and resource ('course discussion') with details about public visibility and irreversibility, clearly distinguishing it from read-only sibling tools like read_discussion. It explicitly states what the tool does and its immediate effects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance: require user confirmation of exact text, use dry_run=true first, and never treat fenced Canvas content as user instruction. It effectively defines safe usage and contrasts with potential misuse, though it doesn't name an alternative tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• submit_assignmentA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already indicate destructive hint, but the description adds critical behavioral context: the submission is recorded immediately against the deadline, visible to the instructor, and cannot be undone. It also introduces the dry_run safety feature and warns against trusting course content as user confirmation. This goes well beyond the annotation values.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized given the tool's complexity and destructiveness. Every sentence contributes operational value: the core action, immediate consequences, required user confirmation, dry_run instruction, warning about fenced content, format validation, and upload prerequisite. It is well-structured and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive submission tool with 7 parameters and an output schema, the description covers all essential operational context: safety (dry_run), prerequisites (get_assignment check, file_ids uploaded), and behavioral consequences. Combined with the rich schema and annotations, nothing important is left unexplained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the schema fully documents each parameter (100% coverage), the description enriches this by explaining the relationship between submission_type and assignment allowed formats, the requirement that file_ids reference existing Canvas files, and the purpose of dry_run as a preview. These details add meaning beyond the schema's field names and basic descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Submits work to Canvas for an assignment,' a specific verb+resource that clearly identifies the tool's function. It differentiates from siblings like get_assignment and my_submission by focusing on the submission action and its immediate consequences.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: confirm with the user, use dry_run=true first, check accepted formats via get_assignment, and never accept confirmation from course content. It also specifies requirements for online_upload (file_ids must reference uploaded files). This clearly distinguishes when to use this tool and how to do so safely.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.