canvas-teacher-mcp
Server Quality Checklist
Latest release: v0.1.0
- Disambiguation3/5
While many tools have clear distinct purposes, there are several clusters with overlapping responsibilities, such as the grading skills (grade, grade-git-program, grade-nb), page builders (assignment-page-builder, git-asmt-page, build_coding_assignment_page), and document formatters (gws-doc, gws-richdoc, gslides-format). Descriptions help differentiate them, but an agent could still mis-select in these areas.
Naming Consistency2/5Tool names mix snake_case (list_courses, get_page) with kebab-case (agenda-wrapup, assignment-page-builder) and even a verb-less noun style (git-asmt-repo, quiz-builder). Verb prefixes are inconsistent (read vs get vs list, create vs build vs add), making the naming pattern unpredictable.
Tool Count2/5With 58 tools, the server far exceeds the typical well-scoped range of 3-15 and even the 25+ threshold for excessive. The large number likely reflects an integrated suite of skills and utilities, but it risks overwhelming agents and increasing selection errors.
Completeness4/5The tool surface covers a comprehensive teaching workflow: assignments, quizzes, pages, modules, announcements, submissions, grading, and Google Docs. Minor gaps exist, such as no delete operations for most resources and no list/get for announcements, but the core lifecycle is well represented.
Average 3.5/5 across 58 of 58 tools scored. Lowest: 1.3/5.
See the Tool Scores section below for per-tool breakdowns.
- No community issues in the last 6 months
- 30 commits in the last 12 weeks
- No stable releases found
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI status not available
This repository is licensed under MIT License.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
Add a glama.json file to provide metadata about your server.
If you are the author, simply .
If the server belongs to an organization, first add
glama.jsonto the root of your repository:{ "$schema": "https://glama.ai/mcp/schemas/server.json", "maintainers": [ "your-github-username" ] }Then . Browse examples.
Add related servers to improve discoverability.
How to sync the server with GitHub?
Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.
To manually sync the server, click the "Sync Server" button in the MCP server admin interface.
How is the quality score calculated?
The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).
Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.
Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).
Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.
Tool Scores
- Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. The phrase 'One module' says nothing about return format, error behavior, permissions, or side effects, making the tool behaviorally opaque.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely brief, but this is under-specification rather than conciseness. It fails to convey any substantive information, so the brevity does not serve the agent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple get-by-ID tool, the description is still inadequate. It does not explain what is returned, how the module is identified, or how it relates to list_modules. With no output schema or annotations, this definition leaves too much to guesswork.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, and the description does not explain 'course' or 'module_id'. The two-word description adds no value beyond the field names, leaving parameter semantics entirely undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose1/5Does the description clearly state what the tool does and how it differs from similar tools?
The description 'One module' is a noun fragment with no verb, so it does not state what the tool does. It could be mistaken for a data definition rather than a retrieval action, and it fails to differentiate from sibling tools like list_modules.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided about when to use this tool versus alternatives. There is no mention of use cases, prerequisites, or exclusions, leaving the agent to infer from the tool name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of disclosing behavioral traits such as side effects, permissions, or return behavior. The description says nothing about any of these, giving the agent zero insight into what invoking this tool entails.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely under-specified, not concise. It is a single sentence but contains no actionable information, failing to front-load any useful content. While it is short, it lacks substance, making it more of a placeholder than a meaningful description.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema and no annotations, the description should clarify what the tool does and what the output represents, but it does neither. The description is entirely inadequate for an agent to understand when and how to invoke this tool, especially among many sibling tools with vague names.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, making parameter semantics irrelevant. Since there are no parameters to explain, the baseline for a 0-parameter tool is 4, and the description does not need to add parameter-level detail. However, the description also does not help contextualize the tool's purpose, which would be beneficial even with no parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose1/5Does the description clearly state what the tool does and how it differs from similar tools?
The description 'The gslides-format skill document' provides no verb or resource, merely restating the tool name in a vague nominal phrase. It fails to indicate what the tool does, how it operates, or what output it produces. This is essentially a tautology with no functional meaning.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines1/5Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool, what problems it solves, or how it relates to sibling tools. The description offers no context about scenarios where this tool is appropriate or when alternatives should be chosen.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose any behavioral traits. It does not state that this is a read-only operation, what the output format is, or whether it requires a course parameter. The description is purely a static noun phrase and entirely fails to address behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short (6 words), which is concise, but it is a fragment rather than a structured sentence. It conveys the general topic but lacks a verb and context. This is not the extreme under-specification of 'Process', but it is still too minimal to be considered well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has one required parameter and no annotations. While an output schema exists, the description does not explain that this tool likely lists groups for a specific course, nor does it clarify the relationship between 'assignment groups' and 'weights'. The description is incomplete for a tool that takes a course parameter and likely returns data.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one parameter, 'course', with 0% description coverage. The description does not mention this parameter or explain its meaning or allowed values. Since schema coverage is low, the description should compensate, but it adds no parameter information whatsoever.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose2/5Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Assignment groups and their weights' is a noun phrase, not a verb phrase, so it does not clearly state what action the tool performs. The name 'list_assignment_groups' suggests listing, but the description fails to confirm this and does not distinguish from sibling tools like list_assignments.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives. The description only states content ('assignment groups and their weights') but does not provide context, prerequisites, or mention of other tools that might be more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of disclosing behavior. It only lists output fields (title, description, question count, points) but does not explicitly state that it is a read-only operation, mention permissions, error conditions, or side effects. This is minimal behavioral disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely short, with no wasted words. However, it is a fragment rather than a complete sentence, and its structure is a simple list. While concise, it sacrifices clarity and completeness, making it only adequate in this dimension.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 required parameters, no output schema), the description still falls short. It fails to clearly state the tool's purpose, provide usage context, or explain parameter roles. The absence of any behavioral or usage details makes it highly incomplete for an agent to select and invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain the parameters 'course' or 'quiz_id'. It adds no meaning to the input schema, leaving the agent with only names and types. The description lists output fields, not input parameters, so it fails to compensate for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose2/5Does the description clearly state what the tool does and how it differs from similar tools?
The description 'One quiz: title, description, question count, points.' does not explicitly state the action of retrieving a quiz. It merely identifies the tool as dealing with a single quiz and lists some fields, which essentially restates the name 'get_quiz' without a clear verb or resource. This makes the purpose vague and does not distinguish it from sibling tools like get_doc or get_assignment.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives. The description lacks any mention of typical scenarios, prerequisites, or comparisons to other tools such as list_quiz_questions or create_quiz. This provides no decision-making support for the agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of disclosing behavior. It only states a defining condition (submissions waiting to be graded) but fails to mention whether it is read-only, how filtering works, what the output contains, or any side effects. This is minimal behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single phrase, which is concise in length, but it is under-specified rather than efficiently informative. It omits critical details and does not earn its place as a useful explanation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with two parameters, no annotations, and an output schema, the description provides almost no contextual completeness. It doesn't explain how the course parameter scopes the result, what 'only_open' affects, or what fields are returned. The description is inadequate for an agent to reliably select and invoke this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, and the tool description mentions neither the 'course' nor 'only_open' parameters. The agent receives no semantic help beyond the parameter names and types, which is insufficient for correct invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose3/5Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Assignments with submissions waiting to be graded' identifies a specific resource type but lacks a verb, so it doesn't clearly state what the tool does (e.g., list, fetch, filter). It is more than a tautology but not a crisp action statement, and it doesn't distinguish from sibling tools like list_submissions or classify_submissions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided about when to use this tool versus alternatives such as list_submissions or get_submission. There is no mention of context, prerequisites, or exclusions, leaving the agent to infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry the full burden of behavioral disclosure. It only mentions that the result includes a description, giving minimal insight into output or side effects. It does not disclose read-only status, permissions, error behavior, or any other operational traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely short and free of filler, but it is under-specified. While brevity is positive, the fragmentary nature means it doesn't earn its place as a complete explanation; it sacrifices clarity for conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool lacks an output schema and annotations, so the description needs to provide more context about what the tool returns and when it should be used. 'One assignment, description included.' is far too minimal to be considered complete for an agent to select and invoke the tool confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain the 'course' or 'assignment_id' parameters. It adds no meaning beyond the parameter names themselves, failing to compensate for the lack of schema descriptions. This is a significant gap for a 2-parameter tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose3/5Does the description clearly state what the tool does and how it differs from similar tools?
The description 'One assignment, description included.' is highly vague. It implies the tool retrieves a single assignment and includes its description, but lacks an explicit verb or complete sentence to clarify functionality. It does hint at a distinction from list_assignments but doesn't clearly state the action.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like list_assignments or read_assignment_instructions. The description does not include any context, prerequisites, or exclusions, leaving the agent to infer usage solely from the tool name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description only lists output fields. It does not disclose whether the operation is read-only, what happens if the course is not found, pagination behavior, or any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence, which is concise, but it omits essential information. It gains some value by naming the output fields, but the brevity comes at the cost of completeness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With one required parameter that is completely undocumented and no usage context, the agent cannot correctly invoke this tool. The presence of an output schema does not compensate for the missing input semantics.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter, 'course', is not mentioned in the description at all. Schema description coverage is 0%, and the description does not compensate by explaining what value should be passed or how to format it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose3/5Does the description clearly state what the tool does and how it differs from similar tools?
The description names the resource (modules) and lists fields, but lacks an explicit verb like 'list' or 'return'. The tool name makes it clear, but the description itself is a noun phrase. It does distinguish from get_module which implies a single module.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is given on when to use this tool versus alternatives like get_module or list_module_items. It does not mention required context such as having a course ID or any prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the behavioral disclosure burden. It only mentions 'active students' and output fields, but does not state that the operation is read-only, how the 'course' parameter affects results, or any response characteristics beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely short and front-loaded, which is efficient, but it omits critical information. It is not properly sized for the tool's complexity, as it fails to address the required course parameter, making it under-specified rather than appropriately concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While an output schema exists, the description lacks essential context about the required course parameter and when to use the tool. It is incomplete for a list operation that depends on a course filter, leaving the agent to guess at the tool's behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema lists a single required parameter 'course' with no description, and the description does not mention it at all. With 0% schema coverage and no compensation in the description, the agent cannot infer the parameter's format or purpose.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose3/5Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Active students: id, name, email.' identifies the resource (students) and a scope (active), but lacks an explicit verb like 'list' or 'return'. It is clear enough that the tool deals with students, but does not fully specify the action, so it is not as strong as a purpose statement.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives such as list_courses or get_course. It does not mention that a course is required or what filters apply, leaving the agent without direction on selecting this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It only mentions that results are ordered, but does not state whether the operation is read-only, how pagination works, what the response structure looks like, or any other behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with a single short sentence, which is appropriate for a simple tool. However, it lacks structure and additional detail that could help the agent, making it somewhat under-specified rather than effectively concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, no parameter descriptions, and a brief purpose, the description is incomplete for proper tool selection and invocation. Even though an output schema exists, the lack of input guidance and behavioral context leaves significant gaps for a tool that requires two parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description provides no explanation of the parameters 'course' or 'module_id'. The schema only lists types and titles, so the agent has no clue what values to provide or how they relate to the module listing.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description 'What is inside a module, in order' clearly indicates that the tool lists the items within a module, and the 'in order' phrase adds specificity about ordering. It distinguishes itself from sibling tools like list_modules or get_module by focusing on the contents, though it could be more explicit about listing items.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. The description does not mention any context, prerequisites, or exclusions, leaving the agent to infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It only notes 'unpublished', but does not specify side effects, required permissions, return values, or whether the operation is reversible. The word 'Create' implies mutation, but no further detail is given.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, using only four words to convey the core purpose. It is front-loaded and free of fluff, though it sacrifices necessary detail for brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations, output schema, and parameter descriptions, the description is inadequate for a complete understanding. It fails to provide usage context, behavioral details, or parameter semantics, leaving significant gaps for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not compensate by explaining the parameters. 'Name' and 'Course' are self-explanatory from their titles, but no additional meaning or format is provided, leaving the description to add no value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create a module') and adds the state 'unpublished', providing a specific verb+resource. It does not explicitly distinguish from sibling tools like add_module_item or list_modules, but the create intent is obvious.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. The description does not mention any prerequisites, contextual triggers, or exclusions, relying solely on the tool name for implied usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full responsibility for disclosing behavioral traits. It only states that the page is created unpublished, omitting details about permissions, side effects, whether existing pages are overwritten, or what the response contains. This is insufficient for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded, with no wasted words. However, it is under-specified for a tool with three required parameters, bordering on a tautology despite being a complete sentence.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, no annotations, and only a four-word description. The tool appears to create a page (likely a Canvas page), but the description fails to explain the return value, how it fits with sibling tools like list_pages/update_page, or any contextual constraints, making it inadequate for a create operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and the description does not mention any of the three required parameters (course, title, body). While the parameter names are somewhat self-explanatory, the description adds no meaning about their expected format, constraints, or how they relate to the page being created.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (create) and resource (page), and adds the 'unpublished' state, which gives it a specific scope. It is distinguishable from siblings like update_page and get_page, though it doesn't explicitly contrast them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool vs alternatives such as update_page or list_pages. There is no mention of prerequisites, use cases, or exclusions, leaving the agent to infer usage from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of disclosing behavioral traits. It merely lists output fields (user id, submitted_at, current score, attempt count) but says nothing about whether the operation is read-only, whether it returns all submissions for an assignment, if there is pagination, or if any side effects or permissions apply. This is a significant gap for a tool that likely reads data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, consisting of one short sentence that is front-loaded with the core purpose. Every word adds meaning about the expected output. However, it is missing important context, so it is not as effective as a slightly longer but more complete description. Still, for conciseness alone, it earns a high score.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that this is a simple list tool with only two parameters and an output schema exists, the description does not need to explain return values. It provides a minimal but acceptable overview of the data returned. However, it omits any mention of the tool's scope (e.g., that it operates per course/assignment) and does not indicate any behavioral details, leaving the agent to infer from parameter names. This makes it marginally adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not compensate by explaining the parameters. While 'course' and 'assignment_id' might be self-explanatory in name, the description does not mention them at all, leaving the agent to infer their roles. The tool name and output fields imply a relationship, but the description adds no semantic value beyond the schema names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Who submitted what: user id, submitted_at, current score, attempt count.' clearly conveys that the tool returns submission data, listing users and their submission details. It is more specific than a bare tautology like 'List submissions' and aligns with the tool's name. However, it does not explicitly name a verb or distinguish from sibling tools like get_submission or classify_submissions, so it falls short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as get_submission or classify_submissions. It does not mention any prerequisites, context, or exclusions. The only implied usage is that the tool lists submissions, but there is no explicit comparison or recommendation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It clearly states the tool creates a proposal, never an applied decision, which is a key behavioral trait. However, it does not disclose side effects, whether it saves data, what happens after approval, or permission requirements. The priority order adds useful context but is incomplete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is brief but the first sentence is an awkward noun phrase ('The rubric the engine reads...') rather than a clear action statement. It is concise, but the structure is not optimally front-loaded for an agent to quickly grasp the tool's purpose. The second sentence clarifies, but the opening could be more direct.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The text covers the proposal nature and priority order, but for a tool with 3 undocumented parameters and no output schema, it leaves many gaps: what the tool actually does with the inputs, what the response looks like, how it integrates with the assignment workflow, and any side effects. It is only minimally complete for an agent to safely invoke.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and the description provides no explanations for the parameters course, code, or grader. The phrase 'for this assignment' hints at context but does not define any parameter meaning. The description fails to compensate for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description implies the tool proposes a rubric for instructor approval by stating 'a proposal, never an applied decision' and mentions the instructor's approval. However, it lacks an explicit action verb like 'proposes' or 'creates', relying on the second sentence for clarity. It does distinguish from siblings like get_rubric by indicating this is a proposal, not a retrieval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. The priority statement explains rubric sources (instruction sheet, Canvas rubric, default) but not usage conditions. It does not mention prerequisites such as needing an existing assignment or that get_rubric should be used for inspecting existing rubrics.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It does not state whether the tool is read-only, what side effects might occur, or what the output format looks like. The phrase 'classified by kind' hints at behavior but gives no details about kinds or error handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, short phrase that is very concise and front-loaded. It avoids unnecessary words, though it reads more as a fragment than a complete sentence, which slightly reduces clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite being a simple single-parameter tool, the description lacks important information about return structure, classification kinds, and edge cases. With no output schema and no annotations, the description is too sparse for an agent to fully anticipate tool behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter 'html' is a string with 0% schema description coverage. The description compensates somewhat by specifying it is 'a block of Canvas HTML,' which adds context beyond the schema. However, it does not explain expected HTML structure, encoding, or limitations, leaving gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description communicates that the tool processes a block of Canvas HTML and returns every link classified by kind, which is a clear statement of purpose. It goes beyond a simple restatement by specifying the input type and output classification, but lacks an explicit verb and does not differentiate from sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool relative to the many sibling tools. There are no mentions of alternatives, prerequisites, or specific contexts where this extraction would be preferred.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden. It adds only minimal behavioral context ('body included', slug's meaning). It does not disclose whether the operation is read-only, what happens on missing pages, authentication needs, or return format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very brief with no wasted words. The two short sentences are front-loaded and readable, though the fragment 'One page, body included' is somewhat telegraphic.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and no annotations, the description should provide a more complete picture. It fails to explain the course parameter, error behavior, or what the response looks like beyond 'body included'. The documentation is adequate for a human familiar with the system but insufficient for an autonomous agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain parameters. It clarifies that slug is the page's URL segment, which adds value, but the required 'course' parameter is completely unexplained. This leaves a significant gap for correct invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description 'One page, body included' combined with the tool name clearly identifies that this retrieves a single page including its body. This differentiates it from list_pages and similar page-related siblings, though it lacks an explicit verb like 'get' or 'retrieve'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The use case is implied: fetch one page with its body, as opposed to listing pages. There is no explicit guidance on when to use this tool versus alternatives, nor any exclusions or mention of prerequisites like course context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It only states the action and due_at format, but does not mention overwriting behavior, validation, side effects, or permission requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise: two short sentences that front-load the action and provide a key format detail. It is not padded, but slightly under-specified.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a batch mutation tool with three required parameters, no annotations, and no output schema, the description does not explain whether all assignments belong to the same course, what happens on failure, or what the return value indicates.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description needs to compensate. It provides useful format info for due_at (ISO-8601 UTC), but gives no semantic guidance for course or assignment_ids beyond the schema titles, leaving ambiguity about identifiers.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and resource: 'Set one due date on several assignments.' This clearly distinguishes it from sibling tools like update_assignment by emphasizing the batch aspect.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives such as update_assignment, nor any mention of prerequisites, appropriate scenarios, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden and does disclose one behavioral trait: publish state is untouched. However, it does not mention partial update semantics, null handling, permissions, or return values, leaving significant gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, no filler. The first sentence establishes the operation, and the second adds a valuable caveat. Concise and front-loaded, though arguably too brief for the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 6 parameters, no annotations, no output schema, and many sibling tools, the description is incomplete. It does not explain which fields are updatable, whether nulls clear fields, or what happens to unspecified fields, so the agent is left with insufficient context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description does not mention any parameters. With schema description coverage at 0% and 6 parameters, the description fails to add meaning beyond the raw schema field names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Update an assignment' which is a clear verb+resource combination, and adds a scope clarification ('Publish state is left alone'). It distinguishes from create_assignment but not from set_due_dates or other update-like tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for modifying assignment fields, but provides no explicit when-to-use guidance, exclusions, or alternatives. The context is clear but the guidance is minimal.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of explaining behavior. It does disclose the returned fields, giving a basic expectation of output, but it does not explicitly state that the operation is read-only or describe limitations such as pagination, ordering, or filtering. The lack of side-effect disclosure is less critical for a list operation but still a gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single compact fragment with no redundant words. It is front-loaded and easily scannable. Every word contributes meaning, making this an appropriately concise and well-structured description.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although an output schema exists, the description does not cover the input context, usage conditions, or list constraints. It repeats what appears to be the output fields but omits the crucial course scoping and any filtering, ordering, or pagination behavior. For a tool with one required parameter and no annotations, this is insufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, and the description does not mention the required 'course' parameter at all. The listed fields refer to assignment output, not to how the course parameter should be supplied or what values it accepts. The description fails to compensate for the absence of schema-level parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly indicates that the tool returns all assignments and lists the included fields (id, name, points, due date, published state). The phrase 'Every assignment' distinguishes it from tools like get_assignment that fetch a single assignment. However, it is a fragment rather than a full sentence and doesn't explicitly mention the course scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit guidance on when to use this tool versus alternatives such as get_assignment or list_assignment_groups. The intended usage is only implied by the tool name and the required 'course' parameter. The description does not state exclusions, prerequisites, or alternative recommendations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description must fully disclose behavioral traits. It only mentions the output (course slugs) and does not clarify that this is a read-only operation, potential authorization needs, or any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence, making it concise and free of filler. However, it is phrased as a noun fragment rather than a complete sentence, slightly reducing clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters, output schema exists), the description is mostly complete for invocation. However, it lacks context about what 'course root' means and does not mention any edge cases or limitations, so it falls short of full completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the schema already fully specifies the invocation. The description adds no additional parameter semantics, but none are needed; baseline score of 4 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states that the tool returns every course slug in the course root, making it clear that it lists all courses. It is distinct from siblings like get_course by indicating 'every' rather than a single resource, though it lacks an explicit verb.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives no guidance on when to use this tool versus alternatives such as get_course or list_students. There is no mention of typical use cases, prerequisites, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description carries the behavioral transparency burden. It does state the scope ('every') and the returned fields, which implies a read-only listing operation. However, it does not mention ordering, pagination, error behavior, or explicitly confirm that no modifications occur.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no fluff. It is front-loaded with the core scope and output fields, though it could be more explicit by using a verb rather than a noun phrase.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return value details are not necessary. However, the description does not explain how this tool fits into the quiz workflow (e.g., after add_quiz_questions or before finalize_quiz) or what course/quiz_id refer to. Given the tool's low complexity, this is minimally adequate but leaves gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, and the description provides no additional meaning for 'course' or 'quiz_id'. The parameter names are somewhat self-explanatory (course and quiz ID), but the description does not explain formats, relationships, or how to resolve them from other tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description identifies the resource (quiz questions) and states the returned fields (id, name, type, points). Although it lacks an explicit verb like 'list', the phrase 'Every question on a quiz' is clear and distinguishes this from siblings like get_quiz or add_quiz_questions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as get_quiz or add_quiz_questions. It does not mention prerequisites, context, or exclusion cases, leaving the agent to infer usage from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full responsibility for behavioral disclosure. It merely states the action without explaining whether the tool is read-only, what it returns, whether it mutates state, or any side effects. The limited behavioral info falls short of what an agent needs to safely invoke the tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words. However, it is slightly under-specified; more context would improve its utility without sacrificing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has no output schema and no annotations, and the description does not explain the return format or how the classification relates to other tools. Given the existence of related tools like list_submissions and needs_grading, the description is too sparse to provide a complete understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no property descriptions, so the description must explain the parameters. It only says 'Split the submissions' without clarifying how course and assignment_id are used, their formats, or constraints. The parameter names are self-explanatory but the description adds no additional semantic value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: it splits submissions into four specific categories (graded, resubmitted, new, not submitted). This is a specific verb-resource pairing that distinguishes it from sibling tools like list_submissions or needs_grading.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies a use case (categorizing submissions) but does not explicitly state when to use this tool versus alternatives such as list_submissions or needs_grading. No exclusions or alternative tool mentions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries the behavioral burden. It discloses that the rubric may not exist ('when it has one') and that there is a precedence relationship with the student-visible rubric. It does not specify return values (e.g., null behavior), error conditions, or permission requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact—two short sentences that efficiently communicate the core purpose and an important caveat. It avoids fluff and front-loads the main point, though the first sentence is a fragment rather than a full declarative sentence.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter getter, the description covers the essential purpose and a key distinction, but it leaves gaps: no output schema, no explicit null behavior, and no instructions on how parameters relate to the rubric lookup. Given the sibling tools and lack of annotations, this is adequate but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the architecture needs the prose to explain the parameters. The description references 'the assignment' but never mentions 'course' or 'assignment_id' explicitly, nor their formats or relationships. The parameter names themselves are straightforward, but the description adds no semantic value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description identifies the resource as 'the assignment's Canvas rubric object' and notes the conditional 'when it has one,' which implies retrieval. It distinguishes this from the student-shown rubric, clarifying scope. However, it lacks an explicit action verb, relying on the tool name.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The second sentence provides a usage guideline: the student-shown rubric 'outranks this' and the assignment page carries its own table, implying this tool is for the stored rubric, not the displayed one. This gives some contextual guidance but does not explicitly name an alternative tool or say definitively when to use this vs. other rubric-related tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It does disclose two behaviors: the assignment is created unpublished and due_at must be ISO-8601 UTC. However, it does not mention permissions, reversibility, validation, or what happens to invalid input, leaving significant gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that front-loads the action and includes only essential information. No waste, every word contributes.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 6 parameters and many sibling tools, but the description is too sparse to provide a complete picture. It does not mention what the return value is, how to reference the created assignment later, or how it fits into the workflow of other tools (e.g., publishing via update_assignment). It is not enough for a create operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, and the description only adds context for one parameter (due_at format). It does not explain the expected format for course (e.g., ID vs name), constraints on points_possible, or the meaning of assignment_group_id. This is insufficient compensation for the schema's silence.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Create' and the resource 'assignment', and adds a key qualifier 'unpublished' which distinguishes it from update or list operations. This is a specific and unambiguous purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like update_assignment or set_due_dates. The 'unpublished' note is a behavioral detail, not usage context. There are no prerequisites or exclusions mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds useful behavioral context by stating items are created 'unpublished' and specifying which parameters are needed for each item_type. However, with no annotations, it does not disclose permissions, side effects, error behavior, or return value, leaving gaps for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded: the first sentence states the core purpose, and the follow-up effectively communicates item_type options and dependencies without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 6-parameter tool with no output schema or annotations, the description covers the item_type logic and conditional parameters well. However, it omits expected format for course (ID vs name), what happens on success/failure, and any permissions or prerequisites, leaving the agent partially informed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description compensates by enumerating valid item_type values and explaining the conditional need for page_url vs content_id. It leaves course, module_id, and title semantics to inference, though those are relatively self-explanatory.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action ('Add an item to a module') and adds the important state 'unpublished'. It identifies the resource and action specifically, but does not explicitly differentiate it from sibling tools beyond the obvious name.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is given on when to use this tool versus alternatives like create_module or list_module_items. The description only provides parameter-level requirements, not tool-level usage context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full transparency burden. It adds a valuable behavioral guarantee that it never returns a secret, clarifying that `canvas_token_env` is a variable name, not the token. However, it does not explicitly confirm the operation is read-only or mention other behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, immediately stating the tool's purpose followed by an important safety note. Every word earns its place; there is no redundancy or irrelevant content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has no annotations and no output schema, so the description must explain both return values and input. It adequately explains the output fields and a security caveat, but fails to clarify the required 'course' parameter, making the description incomplete for reliable invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one required string parameter 'course' with no description, and the tool description provides no information about the parameter's format, acceptable values, or how it identifies the course. With 0% schema coverage, the description fails to compensate, leaving the parameter undefined.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns a course's coordinates (Canvas id, base url, school, GitHub org, output paths), specifying the exact resource and attributes. It distinguishes from siblings like list_courses (which lists courses) and get_doc/get_page by focusing on course-level metadata.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when a course's identifying information is needed, but does not explicitly state when to use it versus alternatives like list_courses or provide any exclusions or prerequisites. The safety warning about secrets adds context but not usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of disclosing behavioral traits. It reveals a critical requirement (confirm='SEND'), which is a safety mechanism. However, it does not describe side effects, whether the action is irreversible, or what the response contains. The reference to 'preview' adds some context but lacks detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two short sentences that front-load the core action and immediately note the critical requirement. No unnecessary words, though the phrase 'which the preview explains' is slightly vague and could be more specific.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the essential action and the confirmation parameter, but for a tool with no output schema and no annotations, it leaves gaps: the effect on recipients, any prerequisites besides confirm, and the relationship to the preview tool are not fully explained. It is minimally viable but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description only mentions the `confirm` parameter (and only its allowed value). The meanings of `course`, `title`, and `message` are left entirely to the schema names. This is insufficient compensation for the lack of schema documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Post an announcement' provides a clear verb and resource, indicating a send/publish action. It does not explicitly differentiate from sibling tools like 'create-announcement' or 'preview_announcement', but the term 'send' implies final delivery, which is distinct enough.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The mention that `confirm='SEND'` is required and that 'the preview explains' gives clear contextual guidance on prerequisites and the intended workflow (likely preview before send). It does not explicitly name the preview tool or list exclusions, but the context is sufficient for basic usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses the core write actions and explains why they are needed (Canvas leaves counts stale). However, it does not mention permissions, idempotency, or any side effects beyond the stated updates.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the main action, and every word earns its place. No fluff or redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 3 parameters with zero schema coverage, no output schema, and no annotations. The description explains why and when to run but leaves intro_html semantics and return behavior entirely undefined, making it incomplete for an agent to invoke without additional hints.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description provides no explanation for any parameter, including the optional intro_html. The phrase 'description summary' might relate to a parameter but is not explicit, leaving the agent without any parameter guidance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states two specific actions: rewrites the description summary and refreshes stale counts, and it names the exact fields (question_count, points_possible). This distinguishes it from sibling quiz tools like create_quiz, get_quiz, and add_quiz_questions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'so this runs after the questions are in,' providing a clear sequencing context. It doesn't mention alternatives or exclusions, but the pipeline position is clear enough to guide usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the burden of conveying behavior. It discloses that the tool returns all pages and lists the exact fields ('title, url slug, published state'), which implies a read-only operation. It does not mention potential ordering, pagination, or error cases, but for a simple listing tool this is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that efficiently conveys the core purpose and key output fields. There is no redundant or filler content; every word adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple (one parameter) and has an output schema, so the description does not need to explain return values in detail. However, the description omits usage guidance and parameter format, and does not mention any behavior like pagination or empty results. It is acceptable for a basic list tool but leaves some gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has only one parameter, 'course', with no description. The schema description coverage is 0%, and the description only says 'in the course' without clarifying whether course is an ID, slug, or name. The description does not compensate for the schema's lack of parameter documentation, leaving the agent to guess how to specify the course.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns every page in a course with specific fields (title, url slug, published state), which corresponds to listing pages. It distinguishes itself from sibling tools like get_page, which retrieves a single page, and create_page/update_page, which modify pages. However, it lacks an explicit verb like 'list' or 'retrieve', so it is not a perfect 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool: when you need all pages in a course, rather than a specific page (get_page) or a different resource type (list_modules, list_courses). However, it does not explicitly state alternatives or exclusions, and there is no direct comparison to sibling tools in the description.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that the quiz is created as 'unpublished' and clarifies parameter formats, but it does not mention return values, permissions, or side effects. This is adequate for a simple creation but leaves gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that immediately states the core action and state, followed by two essential parameter clarifications. No filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 7 parameters, no annotations, and no output schema, the description is too sparse. It omits return value (e.g., quiz ID), prerequisites (e.g., course must exist), and how it fits into the quiz-creation workflow. A simple create tool still needs more behavioral context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description must compensate for all 7 parameters. It clarifies due_at (ISO-8601 UTC) and time_limit (minutes), which are non-obvious, but the other five parameters (title, course, description, allowed_attempts, assignment_group_id) are left to inference from their names. Partial compensation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Create') and resource ('an empty quiz, unpublished'), which distinguishes it from sibling tools like add_quiz_questions and finalize_quiz. The qualifiers 'empty' and 'unpublished' define the exact state of the created object, making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool (create an empty quiz before adding questions/finalizing) but does not explicitly state alternatives or when not to use it. Context from sibling tools (e.g., add_quiz_questions, finalize_quiz) suggests a workflow, but the description itself lacks direct guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It indicates 'reformatting' which implies mutation, but does not clarify whether it modifies documents in place, creates new ones, what side effects occur, or any permission requirements. The behavior is under-disclosed for a tool that can format/write.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the main action, and contains no extraneous information. Every sentence contributes to understanding the tool's purpose and usage, making it highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has no parameters and an output schema exists, but the description remains vague about how the target document is selected and what the exact output or return format is. It lists formatting features but does not fully explain the tool's operational scope, leaving gaps in contextual completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema is empty (0 parameters), so there are no parameter semantics to explain. The schema coverage is trivially 100%, and the description adds no parameter-specific details, which is acceptable given the absence of parameters. Baseline 4 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool as reading and formatting Google Docs via the gws CLI, with examples like heading styles, bold text, and structured content. It is specific about the action and resource, but it does not explicitly distinguish itself from similar siblings like build_rich_doc or gws-richdoc, so it misses full differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides direct usage guidance: 'Use when reading a Google Doc's content, or creating/reformatting a Google Doc with proper heading styles, bold text, or structured content.' This gives clear context for when to invoke the tool, though it does not mention exclusions or alternative tools explicitly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses a meaningful trait (publish state untouched) beyond the schema. However, it omits other behavioral details such as return values, permissions, or side effects expected from a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no redundant words. Every element ('update', 'title and/or body', 'publish state left alone') carries meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is adequate for basic use, identifying the updatable fields and a key behavioral constraint. However, for a write tool with no output schema and no annotations, it lacks information about return values, permissions, or error conditions, making it incomplete for an agent to fully understand the outcome.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It adds context for 'title' and 'body' (updatable fields), but the required 'course' and 'slug' parameters are not explained at all. This is only partial compensation for the missing schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific action ('Update a page's title and/or body') with a distinct resource and verb. It differentiates from sibling tools like create_page and get_page, and the 'Publish state is left alone' note further scopes the behavior.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by focusing on updating title/body, but it does not explicitly reference alternatives or provide when-to-use/when-not-to-use guidance. The publish state exclusion is a minor guideline, but no explicit contrast with create/get tools is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the transparency burden. It explains this is a methodology (the HOW) and that policy lives elsewhere, which adds context beyond the name. However, it does not disclose concrete behavioral effects, side effects, or prerequisites, such as what happens when invoked or what permissions are needed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose and provides useful distinctions (code vs. method, global vs. per-course, policy references) in a compact form. Each sentence earns its place, though some redundancy could be trimmed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and the presence of an output schema, the description adequately covers what the tool is and when to use it. It explains the relationship to the skeleton and per-course skills, but lacks a bit of detail on the expected outcome or workflow steps, though this may be intentionally deferred to the referenced skills.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the schema is trivially complete. The description adds context by explaining that course-specific content comes from per-course skills, but since there are no parameters to document, the description appropriately avoids unnecessary detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool is a 'GLOBAL methodology for building a Canvas ASSIGNMENT/LAB page' from a universal skeleton, giving a specific verb, resource, and scope. It distinguishes itself from per-course skills and sibling builders by framing itself as the global method, though it does not explicitly name alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly says 'Use when building/updating any Canvas assignment or lab page,' providing clear usage context. It also references per-course skills as customizers, implying when those would be preferred, but does not give explicit exclusions or alternatives beyond that.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses the response scope—'body and attachments included'—which clarifies what the agent can expect to receive. While it does not mention side effects or permissions, the read-only nature is strongly implied by the 'get' verb. For a simple retrieval tool, this is adequate transparency, though it could benefit from explicit read-only confirmation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one concise sentence: 'One submission in full, body and attachments included.' It is front-loaded with the core purpose and includes the key detail about body and attachments. No words are wasted, and every element adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple getter tool, the description covers the primary behavior (returning a full submission) but lacks important context. It does not explain the parameter semantics, provide usage guidance relative to sibling tools, or describe the return structure beyond body and attachments. With no output schema, a bit more detail would be helpful, but the tool is basic enough that this is minimally viable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description provides no additional meaning for the three parameters (course, assignment_id, user_id). Although the parameter names are somewhat self-explanatory, the description does not explain how they combine to uniquely identify a submission or what format 'course' should take. This is a significant gap given the absence of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool retrieves a single, complete submission including body and attachments. This distinguishes it from sibling tools like list_submissions, which lists submissions, and classify_submissions, which categorizes them. The verb 'get' is implied by the tool name, making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage: 'One submission in full' suggests this tool is for fetching a specific submission rather than listing all submissions. However, it does not explicitly state when to use this tool versus alternatives like list_submissions, nor does it provide any exclusion criteria or prerequisites. The guidance is implied but not articulated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden of behavioral disclosure. It mentions 'GLOBAL' and that it generates Canvas pages, but does not mention side effects (e.g., whether pages are created or updated, whether existing pages are overwritten), required authentication, configuration failure behavior, or idempotency. This is a significant gap for a tool that produces real Canvas artifacts.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded: the main action and target resources appear in the first sentence, then supporting details (config-driven, triggers) follow. Every sentence adds value, and the trigger list is concise and actionable without being redundant.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description gives a good overview but omits operational details: it does not explain how the tool identifies the module, how the week is determined, or what happens if configuration is missing. The presence of an output schema helps, but for a tool with no explicit parameters and non-trivial behavior, more context is needed for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, and schema coverage is trivially 100%. Per the rubric, the baseline for 0 parameters is 4. The description adds context by mentioning 'course_slug -> course_config' and the concept of being config-driven, which hints at how inputs might be resolved even though there is no explicit parameter list.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description begins with a specific verb 'generate' and names the exact resources: 'weekly Agenda (week start) and Wrap-up (week end) Canvas pages for a module'. It also clarifies data source ('from the same module data'), which distinguishes it from sibling tools like create_page or module-overview-page.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear trigger phrases ('week N agenda', 'wrap-up for week N', 'update the agenda', 'make the week 3 agenda/wrapup') indicating when to invoke the tool. It does not explicitly state when not to use alternatives, but the context is strong enough for an agent to recognize appropriate invocations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the burden of behavioral disclosure. It adds useful context that the rubric TEXT is derived from the notebook itself and only the weight split is a default. However, it does not disclose side effects such as whether the page is created as a draft, published, or if existing pages are overwritten, nor does it mention authentication or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three short sentences, front-loaded with the core purpose, followed by two informative caveats. Every sentence earns its place and there is no redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a fairly complex tool with 5 parameters, a nested opaque lab object, no output schema, and no annotations. The description provides key caveats and a pointer to the required skill, but does not explain how course, due_at, points, and publish_to_canvas interact, nor what the return/result looks like. It is adequate but leaves important gaps for reliable invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It clarifies the opaque lab parameter by saying it 'describes the notebook' and hints at rubric/weight behavior. Other parameters (course, due_at, points, publish_to_canvas) are left to self-explanatory names and defaults, but the internal structure of lab is not detailed, leaving a meaningful gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Build'), the resource ('Canvas page'), and the specific scope ('Colab/Jupyter notebook assignment'), distinguishing it from the sibling tool build_coding_assignment_page. The verb+resource+scope structure is specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides a clear prerequisite ('Read the nb-homework-create skill first') and context about rubric derivation, which helps the agent know how to prepare. However, it does not explicitly name alternatives or state when not to use this tool, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It clearly discloses the key behavioral trait: "Sends nothing." It also explains the output behavior (render and instructions to send). While it doesn't mention other side effects, the primary safety concern (unintentional sending) is explicitly addressed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences with no wasted words. It front-loads the core function (render) and immediately adds the critical caveat (sends nothing). Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple preview tool with no output schema, the description adequately covers the essential behavior. It explains what it does (render) and what it doesn't do (send). It doesn't describe the return format in detail, but that is not critical given the tool's simplicity and the self-explanatory parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and the description adds no parameter-level detail. The parameter names (course, title, message) are self-explanatory, but the description does not clarify expected formats, types, or relationships beyond what the names imply. It fails to compensate for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool renders what would be sent and explicitly says "Sends nothing," distinguishing it from sibling send tools. The verb "Render" and the object "what would be sent" make the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies this tool is for previewing an announcement without actually sending it, but it does not explicitly state when to use it instead of send_announcement or create-announcement. No exclusions or alternatives are named, so the usage context is only implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the transparency burden. It discloses key behaviors: returns per-student scores and evidence, posts nothing, writes no comment, and saves artifacts. This adds significant context beyond a generic 'run' description, though it doesn't mention permissions or reversibility.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with a clear lead sentence and a precise clarification. No redundancy; every clause adds information about behavior or return value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the main purpose, return values, and non-actions (no posting, no comments). It lacks parameter details and specifics about artifact storage, but given the tool's complexity, the high-level context is well covered. The absence of an output schema is compensated by the return-value explanation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not mention the parameters (course, code, rubric) at all. The agent must infer their meaning from names alone. The description fails to compensate for the lack of schema documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action: run Stage A to collect submissions, score machine-scorable items, and save artifacts. It also explicitly distinguishes itself by noting it posts nothing and writes no comment, separating it from sibling tools like post_grades.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context: this is the automated scoring stage before Stage B (wording) and before posting, which is a separate authorised step. It implies when to use the tool and when not to, but doesn't explicitly name an alternative tool or provide explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of behavioral disclosure. It explains that the tool 'Owns the WHAT' and delegates HOW to L2 and course coordinates to L3, adding architectural context. However, it does not describe concrete invocation behavior, side effects, or return format beyond the existence of an output schema. This is a moderate disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but each sentence adds value: it defines the scope, the WHAT pipeline, delegation to L2/L3, and usage instructions. It is front-loaded with 'GLOBAL, language-agnostic L1 abstract.' No wasted words, though the chain 'audit → instruction-fidelity harness → prove 100/100 → ship the Starter → gates' is compact and may require interpretation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's abstract nature, no parameters, and presence of an output schema, the description sufficiently situates it within the broader system (L1 vs L2 vs L3, course wrappers). It does not explain what a caller should expect on invocation, but the output schema covers that. The description is complete enough for an agent to understand its place and usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters and the schema coverage is 100% (vacuously), so the baseline is 4. The description does not need to add parameter details; it correctly focuses on the tool's role. No additional parameter semantics are required.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool as a global, language-agnostic abstract for building and testing GitHub coding-assignment repos, distinguishing it from siblings by making explicit its L1 role and delegation to L2/L3. It lacks a concrete verb+resource phrasing but communicates scope and intent effectively.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage guidance: 'Use as the shared core every course's git build/test wrapper points to' and 'NEVER re-implement this per course.' It also names the alternatives (L2 language skills, L3 course wrapper), making it clear when to use this tool versus others.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that all Canvas quiz API calls go through quiz_builder.py, dispatches questions by type, and defers git-program questions to git-homework. However, it does not mention whether updates overwrite or merge existing questions, permission requirements, or other side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description has five sentences with redundancy: 'building/updating', 'Create a quiz from nothing OR modify an existing one', and 'making/updating' repeat the same core idea. It is clear but could be more tightly written.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with zero parameters and an output schema, the description covers the essential functionality: building/updating quizzes, dispatching question types, and the API-call policy. It does not explain the 'description summary' or relationship to finalize_quiz, but it is adequate for a methodology-level tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so description adds little parameter-specific meaning. Baseline for 0 params is 4; the description does not need to explain parameters and instead focuses on scope and behavior.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool as a global methodology for building/updating Canvas CLASSIC quizzes, covering both question items and the description summary. It distinguishes from sibling tools like create_quiz and add_quiz_questions by emphasizing its global, dispatch-based approach and the use of quiz_builder.py.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states 'Use when making/updating any Canvas quiz's questions or details' and covers both creating from scratch and modifying existing quizzes. It also notes that git-program questions defer to git-homework, providing an alternative path, but does not name specific sibling tools to avoid.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description bears the full burden of disclosure. It reveals that the tool fails fast if the `gws` CLI is missing, and that an omitted `folder_id` places the doc in the default location with a follow-up response. This is valuable, but it doesn't cover other behavioral traits like permissions or the full return shape, so a 4 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded with the main purpose. Each sentence earns its place: one for purpose, one for parameter semantics, and one for the CLI prerequisite and failure mode. No fluff or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 3-parameter tool with no annotations and no output schema, the description covers the core purpose, explains two of three parameters, states the environment requirement, and hints at the response behavior. It could be more explicit about the exact return value, but it provides enough context for an agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, but the description compensates by explaining `blocks` as the generator block form and pointing to the `gws-richdoc` skill, and `folder_id` as a Drive folder with default-location behavior. The `name` parameter is left to inference from its schema title, but overall the description adds meaningful meaning beyond raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific action ('Build a formatted Google Doc') and lists visual elements (banners, boxes, callouts, coloured code blocks), making the tool's purpose clear. It does not explicitly differentiate from sibling tools like gws-doc or gws-richdoc, so it stops short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives concrete guidance: how to fill `blocks`, what happens when `folder_id` is omitted, and a hard prerequisite (the `gws` CLI) with failure behavior. It does not mention alternatives or when-not-to-use this tool, but it provides clear enough context for invoking it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that the tool returns the exact student-facing assignment and emphasizes its authoritative role for grading. It does not discuss errors or side effects, but the behavioral context provided is meaningful.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact, with two sentences that deliver the core purpose and usage context without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a straightforward read tool, the description sufficiently covers what is returned and when to use it. It lacks return format details and parameter explanation, but overall it provides enough guidance for an agent to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema offers no descriptions for 'course' or 'code', and the description does not clarify their meaning or format. Since schema description coverage is 0%, the description should compensate but does not.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies that this tool retrieves the assignment exactly as the student saw it, listing the included components (description, rubric, due date, attachments). This emphasis on the student-facing view distinguishes it from sibling tools like get_assignment or get_rubric.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The second sentence explicitly states that grading should be based on the student's view and that this tool should be used before any scoring. This provides a clear usage context, though it does not explicitly name alternatives or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the transparency burden. It clearly implies a read-only operation via 'Read' and adds the path base constraint, but doesn't disclose error behavior, file type handling, or other edge cases. Minimal but adequate for such a simple tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences, front-loaded with the action and examples. Every word earns its place, with no unnecessary detail or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter read tool with an output schema, the description covers the essential 'what' and 'where' sufficiently. It could mention when not to use it or reference alternatives, but the simplicity of the tool makes this a minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema only defines 'path' as a string. The description adds crucial semantics by stating the path is relative to CourseGlobalWorkflow/ and provides examples. This compensates for the 0% schema coverage and gives the agent actionable guidance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with a specific verb ('Read') and resource ('workflow document'), backed by concrete examples (GRADING.md, Access/Canvas.md). This distinguishes it from sibling tools like get_page or get_course that operate on different resources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear contextual usage: this is for reading workflow documents, with the path relative to CourseGlobalWorkflow/. It doesn't explicitly mention alternatives, but the examples and path constraint make the intended scope obvious.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the transparency burden. It discloses that the actual grading runs through the engine (`core.grade` → `graders/nb.py` + `nb_inspect`) and that the skill is not a standalone driver, which is key behavioral context. It doesn't mention side effects or data changes, but as a sub-skill method, this is less critical.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a concise paragraph of four sentences, all of which add value: it identifies the role, the dispatch condition, the implementation path, and the master reference. It is slightly dense with backticks and file paths, but still efficient and front-loaded with the key purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description provides sufficient context for a sub-skill: what it does, when it runs, how it executes, and where to find the master skill. It doesn't detail the 'concrete steps' but points to consolidated references, which is likely enough for the agent to follow. The empty schema and presence of an output schema mean the description doesn't need to explain return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline is 4. The description doesn't need to explain parameters, and it doesn't attempt to. The schema is empty, so there is no gap to compensate for.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool as the notebook-specific grading sub-skill for the global `grade` skill, distinguishing it from siblings like `grade` and `grade-git-program`. It specifies the resource (Colab/Jupyter notebooks) and the action (grading, inspection), though it phrases it as a 'SUB-SKILL' rather than a direct verb phrase.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage context is explicit: 'When an assignment's submission is a Colab/Drive notebook, `grade` dispatches HERE.' It also states that this is not a standalone driver, so the agent knows to invoke it only within the larger grade workflow. This gives clear when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of behavioral disclosure. It discloses that it loads rules and detects the active course, and mentions precedence behavior. However, it does not explicitly state whether the tool is read-only or has side effects, leaving some ambiguity for an agent deciding to invoke it.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with 'Generic session orientation,' and every sentence adds meaningful information. It is concise, well-structured, and free of filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (0 parameters, output schema present), the description covers the essential aspects: what it does, when to use it, and the precedence rule. It is complete enough for an agent to decide when and how to invoke it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline is 4. The description adds context about the 'current working directory' serving as an implicit input, which is useful beyond the empty input schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with specific actions: 'Loads account + course global rules' and 'detects the active course from the current working directory.' It also distinguishes itself as 'Generic session orientation' versus project-specific orientation, matching the sibling context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear usage context: it is a generic orientation tool, and for project-specific cases it instructs to 'run from inside a project dir' where the per-course readytogo.md takes precedence. This tells the agent when to use it, though it does not explicitly name alternative tools as exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of transparency. It reveals that the tool delegates to git_page, is a child of git-asmt, and that it must fill an 'asmt dict' with detailed fields. It also discloses the 'Canvas side ONLY' constraint, though it doesn't mention side effects or permissions. Overall, it provides more behavioral context than typical descriptions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, dense sentence that packs in purpose, delegation, fields, usage, and constraints. It is front-loaded with the main action and uses punctuation to structure ideas. It could be slightly more concise, but each piece adds relevant context, so it earns a strong score.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and an output schema exists (though not shown), the description covers the what, how, and constraints. It explains the workflow (fill dict, call git_page), language-agnostic behavior, and the Canvas-only scope. Some internal references (e.g., 'L1', 'L3 course wrapper') are cryptic without deeper system knowledge, but for a tool in this ecosystem it is reasonably complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0 parameters, so baseline is 4. The description adds meaning about the internal 'asmt dict' and the call to git_page(course_slug, asmt), but since there are no actual parameters to document, it cannot go higher. The schema coverage is 100% by virtue of having no parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'build the Canvas ASSIGNMENT PAGE for a git coding assignment via the global git_page'. It also distinguishes from siblings by specifying 'Canvas side ONLY' and referencing its parent 'git-asmt', setting it apart from the repo-side tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly says 'Use to make/renew any git-assignment Canvas page', which gives a clear use case. It also indicates that field selection comes from the L3 course wrapper's curriculum, and clarifies this is for the Canvas side only, though it doesn't explicitly name alternatives or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the transparency burden. It discloses the execution path (core.grade → graders/gh), grading strictness differences (quiz vs homework), and the current STUB status. However, it does not explicitly mention potential side effects or whether it writes grades to a gradebook.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and densely packed with valuable information, including purpose, scope, strictness policy, execution path, and status. While the heavy use of jargon and capitalization may reduce readability, every sentence adds meaningful detail and nothing is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and the tool's moderate complexity, the description covers the essential functional context: what it grades, how it executes, the grading policy, and its stub status. It lacks explicit side-effect details but is otherwise complete for an agent to select and invoke the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and the schema coverage is 100%, so there are no parameter semantics to add. The description appropriately omits parameter details, aligning with the baseline for tools with no parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool as 'Grading SUB-SKILL for git-program assignments AND git-program quiz questions', stating it is the type-specific plug-in of the global grade skill and analogous to grade-nb. This makes the specific verb, resource, and scope unambiguous, effectively distinguishing it from sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It states it is the type-specific plug-in for git-program items and handles both assignments and quiz questions, implying when to use it. However, it does not explicitly say 'use this instead of grade' or provide exclusions for tools like grade-nb, though the analogy hints at the distinction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden of safety/side-effect disclosure. It explicitly states 'Returns the rendered HTML; with `publish_to_canvas` it also creates the Canvas assignment,' disclosing the conditional write behavior. It doesn't cover permissions/errors/rate limits, but for a build-and-optionally-publish tool this is meaningful transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded: purpose sentence first, then the assignment payload, then return/side-effect. Every sentence adds information, and the skill pointer prevents duplicating a lengthy field/section spec.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 6 params, nested objects, and no output schema or annotations, this is fairly complete: it explains the return value, the optional publishing side effect, the assignment payload's shape, and points to the `git-asmt-page` skill for ordering/field details. It could add semantics for rubric weights or due dates, but those are reasonably inferable from schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description compensates for the most complex parameter by enumerating `assignment`'s fields and its two modes (`prototype`/`params`/`returns` vs `input`/`output`/`expected`). It also explains `publish_to_canvas`'s effect. The remaining params (`course`, `points`, `due_at`, `rubric_weights`) are not described, but their names/schema types make them self-evident.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Build the Canvas page for a programming assignment whose starter repo is on GitHub' — a specific verb, resource, and qualifying context. It distinguishes itself from siblings like build_notebook_assignment_page by scoping to Git-backed programming assignments, and mentions optional publishing to Canvas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives clear context: this is for programming assignments with starter repos on GitHub and instructs the agent to 'Read the `git-asmt-page` skill before filling it.' It does not explicitly name alternatives or when-not-to-use, but the GitHub-repo condition and skill reference provide adequate guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full transparency burden. It discloses the dispatcher behavior, the sub-steps (solution/tests, prove 100/100, Starter, org-hub registration, Canvas page via git_page), and the ordering. It does not mention auth, failure modes, or side-effect reversibility, but the orchestration details are substantive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single dense paragraph with no filler. Every clause adds value: purpose, child routing, language-agnostic nature, configuration hierarchy, and trigger phrase. The structure is front-loaded with the tool's role.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters, no annotations, and an output schema, the description provides a solid high-level map of the tool's responsibilities and integration with child tools. It does not explain prerequisites like course validity or org-hub setup, but for a dispatcher with this sibling set the coverage is sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema is empty, so the baseline is 4. The description adds useful context by mentioning the natural-language '<CODE>' input, helping the agent understand that parameters are not structured schema fields but are derived from the user command. It does not define the format or constraints of <CODE>, but that is acceptable for a dispatcher with no formal parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies this tool as a dispatcher/entry point for creating a GitHub coding assignment end-to-end, with an explicit 'Use when the user says...' trigger. It distinguishes itself from its children by naming git-asmt-repo and git-asmt-page and explaining the routing relationship.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a clear usage context: 'Use when the user says make/build/create assignment <CODE>'. It also explains the execution order (repo-first then page) and references the child tools, though it does not explicitly list exclusion cases or compare with all sibling alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It outlines the multi-stage workflow and mentions the 'post-gate' and 'instructor go' requirement, which hints at approvals and side effects. It also references bundled contracts and policies. It does not dive into potential safety issues, permissions, or failure modes, but for a coordinator tool, the disclosed process is fairly transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long and dense, but every sentence contributes unique information: the global scope, the stage sequence, the bundled artifacts, and the runnable driver. It is front-loaded with the core purpose, though the later sentences become somewhat run-on and list-heavy, making it less crisp than a 5 would require.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's high complexity as a meta-orchestrator, the description is remarkably complete. It specifies the ordered stages, the bundled policy modules, the engine contracts, and the runnable driver. It also includes the recommendation to invoke before any grading work. An output schema exists, so return value details need not be explained. The high-level runbook is sufficient for an agent to initiate the workflow and know where to find details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, and schema coverage is vacuously 100%. Per calibration, a zero-parameter tool receives a baseline of 4. The description adds no parameter-specific information because there are no parameters to explain, and no further clarification is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb+resource: it is 'GLOBAL step-by-step grading skill' for 'grading ANY course's Canvas assignment OR quiz/exam through the shared grade engine'. It clearly distinguishes itself from siblings by being the 'ONE ordered procedure' and by naming the bundled components (GRADING.md, SKILL.md, CourseGlobalWorkflow/README.md, GradingEngine/*) and the runnable driver grade_skill.py.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Invoke BEFORE any grading work — never grade ad-hoc', which tells the agent when to use this tool. It also describes the full sequence (Stage A → Stage B → report → self-challenge → POST), making the use context clear. However, it does not explicitly name alternative tools or state when not to use it, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses several behavioral traits: GLOBAL scope, config-driven, flexible data categories, renderer module, and slide-plan dependency. However, it does not disclose whether 'build' creates/updates a Canvas page, overwrites existing content, or requires specific permissions, leaving side effects unclear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but each sentence adds meaningful context: purpose, flexibility, implementation details, and triggers. The implementation sentence about the renderer is slightly technical but reinforces the config-driven nature, so no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
A config-driven build tool with no params and an output schema, but the description still needs to explain its internal structure and dependencies. It covers the overview structure, flexible section blocks, renderer/config, slide-plan alias, and triggers. Missing explicit disclosure of side effects (create vs update) and prerequisites, so not a perfect 5.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema is empty (0 parameters), so baseline is 4. The description adds value beyond the schema by explaining that the tool is config-driven and uses course_config.load(<course_slug>), and by listing trigger phrases that help the agent know when and how to invoke it without explicit parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'build a module OVERVIEW page (the representative page at the top of a Canvas module)' with a specific verb and resource. It also distinguishes from sibling tools like create_page or get_module by focusing on the overview page and providing trigger phrases like '대표 페이지'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit triggers ('module overview page', 'make the overview/representative page for module N') and an explicit alternative/dependency: 'For per-slide breakdowns it uses the slide-plan skill.' This tells the agent when to use this tool vs slide-plan.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description takes on the full transparency burden and delivers: it discloses hidden per-cell anchors, pre-filled answer cells, section pins, essay reflection, and the student guidance it bakes in. It also notes the mirror-image relationship with the grader. It stops short of detailing side effects like file creation or permissions, but the disclosure is strong.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two dense sentences that front-load the core purpose and then pack in the key structural behaviors. Every clause earns its place, and there is no fluff or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and the presence of an output schema, the description covers the essential context: what the tool creates, how it structures the notebook, and how it relates to `grade-nb`. It could improve by mentioning when to choose this over `build_notebook_assignment_page` or other notebook-related tools, but overall it is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so schema coverage is trivially 100%. The description does not need to explain any input meaning, and the baseline for 0 parameters is 4. No additional parameter semantics are required.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource: 'Create/prepare an NB (Colab / Jupyter) homework TEMPLATE' and explains its grading purpose. It distinguishes itself from siblings by explicitly pairing with `grade-nb`, making its niche clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context that it is 'Global' and pairs with `skills/grade-nb`, implying it is the template-creation companion to the grading skill. However, it does not explicitly list alternatives or state when not to use it, leaving some room for ambiguity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the return value ('section tuples'), how it integrates with module-overview-page's make_page, and the transformation behavior. It does not mention side effects or permission requirements, but for a non-mutating sub-skill this is likely not critical. The description adds useful behavioral context beyond the empty input schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: it starts with the core function, then describes the return value, usage context, and trigger phrases. Each sentence adds value and there is no fluff. The front-loaded core function makes it easy for an agent to quickly grasp the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema (not shown but indicated) and zero parameters, the description provides sufficient context: what it does, when to use it, what it returns, and how it fits with module-overview-page. It covers the essential behavioral and integration details needed for selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters (empty object), so there are no parameter details to document. The description appropriately focuses on the tool's purpose and output rather than parameters, earning the baseline score of 4 for no-param tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states what the tool does: 'turn a multi-slide module into one SECTION per slide' and explicitly mentions the output ('Returns section tuples that module-overview-page's make_page consumes'). It also implicitly distinguishes itself from sibling module-overview-page by positioning as a sub-skill with a specific transformation role.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives an explicit use case: 'Use when a module has SEVERAL slide decks and you want each slide's content summarized next to the tasks derived from it.' It also provides trigger phrases. However, it does not explicitly state when NOT to use it or name alternative tools, though the sub-skill relationship to module-overview-page provides some context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the upsert behavior (a question with the same `name` is updated in place) and details the exact structure for all four question types. It does not mention broader side effects or permissions, but the given behavior is substantial.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and well-structured: a single introductory sentence with the upsert rule, followed by a concise list of question types and their fields. Every sentence adds value, and there is no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having no annotations or output schema, the description provides all necessary information to construct valid arguments: the upsert rule, the four question types with their required fields, and the note that `text` is HTML. The `course` and `quiz_id` parameters are self-explanatory. The description is complete for practical use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema only provides parameter names with no descriptions. The description fully compensates by detailing the `questions` parameter with per-type field specifications (choices, correct, etc.), which is essential because the schema allows additional properties and lacks structure. This goes well beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Add or update questions', which clearly specifies the verb (add/update) and resource (questions). It further distinguishes itself by describing the upsert-by-name behavior, separating it from sibling tools like list_quiz_questions or create_quiz.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly implies this tool is for creating or modifying questions in a quiz, which sets a clear context. It does not explicitly name alternatives or state when not to use it, but the purpose is unambiguous and no exclusions are needed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly states that course coordinates are loaded from config (nothing hardcoded), that it reads a module and summarizes contents, and critically that it is 'PREVIEW-first' and sends emails only on explicit approval. This is exceptional transparency for a tool that can send emails.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single but dense paragraph, front-loaded with the core purpose and usage. Every sentence earns its place, covering trigger phrases, behavioral details, configuration, safety, and invocation. No fluff or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero annotations and an empty schema, the description covers purpose, usage, behavior, and invocation reasonably well. The main gap is a mismatch between the described required arguments (course_slug, week/module id) and the empty input schema, which could confuse an agent even though the description is explicit about what should be passed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero properties, but the description adds meaningful invocation guidance ('Invoke with the course_slug + week or module id'), which compensates for the empty schema. It does not provide formal parameter names or types, but the baseline for 0 parameters is 4, and the description enhances that with practical instructions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('generate') and identifies the resource ('weekly Canvas announcement') with clear scope ('from a module's contents for ANY course'). It distinguishes itself from sibling tools like preview_announcement and send_announcement by framing a config-driven, global workflow that produces the announcement and manages the send flow.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly states when to use the tool ('Use when the user says make/write the weekly announcement, week N announcement, or announce this week'), which is strong. However, it does not explicitly mention when not to use it or name alternative tools (e.g., preview_announcement for just preview), so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden. It discloses the workflow (parses into quiz_builder's JSON, renders preview HTML, delegates the write to quiz-builder) and notes that per-question points are an input. However, it does not clarify whether the write happens automatically after preview or requires further confirmation, nor does it mention permissions or side effects. Minor gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is slightly long but every sentence contributes distinct information: purpose, process, features, usage, and triggers. It is well-organized and front-loaded. The trigger list adds length but improves practical usability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that there is an output schema and no parameters, the description is fairly complete. It covers the input format, workflow, supported item types, and how it relates to quiz-builder. It does not mention edge cases or failure modes, but this is not critical given the output schema exists.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema is empty, so the baseline is 4. The description adds extra semantics by stating 'per-question points are an input,' which reveals an implicit input mechanism not captured in the schema. This is valuable for an agent.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states the exact function: converting a publisher's plain-text question bank into a Canvas CLASSIC quiz. It specifies the source formats (Liang/Pearson test bank, .txt/.docx), the process (parse, preview HTML, delegate to quiz-builder), and the supported item types. This clearly distinguishes it from sibling tools like quiz-builder or create_quiz.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly provides usage context: 'Use when the user has a chapter test-bank .txt (or .docx text) and wants it as a Canvas quiz.' It also lists concrete trigger phrases, including a non-English one, which helps an agent recognize when to invoke this tool versus alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses key behavioral traits: the tool is deterministic ('DETERMINISTICALLY'), it relies on a fixed generator ('via the fixed generator build.py'), and it mandates using build.py while forbidding hand-authoring HTML. It does not discuss side effects, permissions, or reversibility, but for a zero-parameter doc creation tool with an output schema, this is adequate coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and well-structured: the first sentence states the core function and determinism, the second gives the usage trigger, and the third delivers a critical implementation rule. It wastes no words, and the slight redundancy between 'DETERMINISTICALLY' and 'ALWAYS run build.py' is acceptable for emphasis.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has no parameters, an output schema exists, and the description provides clear purpose, usage context, and behavioral instructions. It also references the specific document style and the deterministic generator, giving the agent enough information to select and invoke the tool correctly. The presence of sibling tools like build_rich_doc does not create ambiguity because the description names the specific request triggers.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline is 4. The description adds no parameter-specific details, but none are needed. The input schema is empty, so there is nothing to clarify. The description's focus on output behavior compensates for the lack of parameter discussion.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Create a rich-format Google Doc' with a specific list of formatting elements (banner, color section boxes, navy step badges, etc.), making the tool's purpose clear and distinct. It also distinguishes this tool from generic doc tools by referencing 'gws-richdoc / rich-gdoc' and 'polished, visually-structured Google Doc', and notes the deterministic method via build.py.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit when-to-use guidance: 'Use when the user asks for a "gws-richdoc / rich-gdoc" doc or a polished, visually-structured Google Doc.' It also gives a clear process directive: 'ALWAYS run build.py — never hand-author the HTML' and explains the reason ('that is what made every session look different'), effectively steering the agent away from an alternative approach.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does so thoroughly: it discloses that canvas_url 'writes the config with an empty token and reports the file', warns that passing token means it 'stays in this conversation's record', and notes optional overrides. These side effects and privacy implications go well beyond the input schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact but information-dense, front-loaded with the main purpose and then organized into clear call modes. Every sentence adds value, and the line breaks improve readability without padding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's multi-mode setup workflow, it covers the main invocation paths, side effects, and safety notes. The output schema exists to document return values, so not describing them is acceptable. It could add a bit more about expected sequence or error behavior, but it is largely complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, yet the description assigns meaningful roles to all five parameters: canvas_url registers a school, token is an optional credential with a privacy caveat, course_url registers a course, and slug/course_dir are optional overrides. It fully compensates for the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a clear directive 'Set this install up, one step per call' and then details distinct actions: checking status with no args, registering a SCHOOL via canvas_url, and registering a COURSE via course_url. This specific verb+resource framing distinguishes it from sibling content/course tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit invocation scenarios: call with no arguments to inspect state, pass canvas_url to register a school, pass token only if the instructor prefers, and pass course_url for a course. It lacks an explicit 'do not use when' statement or named alternative, but the conditional token guidance and one-step-per-call constraint provide solid usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully carries the burden of behavioral disclosure. It reveals key traits: default is a dry run, requiring dry_run=false to actually write; fix mode is the only mode that deletes a comment and only for the current attempt; and the tool aborts if unread evidence exists. These are critical behaviors beyond the basic write operation, making the tool's runtime behavior clear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise yet dense, using four well-structured sentences. Each sentence adds unique value: the core purpose, the input requirement, the dry-run behavior, the fix mode caveat, and the abort condition. It is front-loaded and free of waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the essential context: the expected input state (POST-READY, not Stage-A), the default dry-run safety, the fix mode's side-effects, and the abort condition. Since an output schema exists, the absence of return-value details is acceptable. The description is fully sufficient for an agent to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides only names, types, and defaults with 0% description coverage. The description compensates excellently by explaining grades_json as the POST-READY file, dry_run as the default behavior, and fix as the corrective mode for earlier mis-grades, including the deletion constraint. This gives full semantic meaning to all three parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Write the rendered grades and comments to Canvas.' The verb 'write' and resource 'rendered grades and comments' are specific, and 'Its own step, asked for on its own' distinguishes it as a standalone posting step, differentiating it from grading tools like grade or grade-nb.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides strong usage context: it is a standalone step for posting rendered grades, expects a POST-READY file, and refuses Stage-A files. It also explains the dry-run default and fix mode. However, it does not explicitly name alternative tools or state when NOT to use this tool, so it misses the 'explicit exclusions/alternatives' criterion for a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
GitHub Badge
Glama performs regular codebase and documentation scans to:
- Confirm that the MCP server is working as expected.
- Confirm that there are no obvious security issues.
- Evaluate tool definition quality.
Our badge communicates server capabilities, safety, and installation instructions.
Card Badge
Copy to your README.md:
Score Badge
Copy to your README.md:
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/Canvas-LMS-MCP/canvas-teacher-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server