Server Quality Checklist

Tool Scores

• search_departmentsB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already establish that this is a read-only, idempotent, non-destructive operation. The description adds that matching is keyword-based, which implies partial matching behavior, but does not disclose result limits, pagination, or exact match semantics beyond what annotations cover.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words, and the examples are directly useful for understanding the expected inputs.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity, two required string parameters, and annotations covering safety, the description is minimally adequate. However, with no output schema and no parameter descriptions, it leaves the return shape and the exact format of 'divisions' unspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It maps 'term' to a keyword and 'divisions' to a division code, with examples 'computer' and 'ARTSC', but does not clarify whether 'divisions' accepts a single code or multiple codes (despite the plural name), nor the expected string format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb (search), resource (departments), and filtering dimensions (keyword and division code), with concrete examples for the latter. It is clear what the tool does, though it does not explicitly distinguish itself from the sibling get_divisions, which likely returns unfiltered division data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through 'Search departments by keyword and division code', but gives no guidance on when to use this tool versus alternatives such as get_divisions or get_reference_data. There are no exclusions or prerequisite conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_course_titlesB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint and destructiveHint=false, so the safety profile is covered. The description adds one genuinely useful behavioral note (thresholds are upstream tuning, not pagination), but says nothing about result shape, ranking, or whether the match is prefix/substring.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences with the core action front-loaded and no filler. The second sentence is a targeted disambiguation, so every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, five undocumented parameters, and no annotation gaps explicitly addressed, the description leaves the caller guessing about input formats and what autocomplete returns (suggestions? ranked list?). It is adequate for a simple read-only lookup but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description carries the full burden. It partially compensates, clarifying that divisions/sessions select the autocomplete scope and that thresholds are tuning knobs — but it never explains term vs. course code/title, nor the expected format of divisions/sessions (single value vs. list).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb and resource ('Autocomplete a course code/title') and lists the required inputs. It does not, however, distinguish this tool from the sibling search_courses, which an agent would plausibly consider for the same intent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It implies prerequisites by stating the tool needs 'a division code and a current session ID', which quietly points at get_divisions and get_current_sessions. It never states when to prefer this over search_courses or when it should not be used.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_reference_dataA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety profile is covered. The description adds that the tool surfaces four specific reference categories, but says nothing about return shape, caching, or how values are keyed. Slight added value over annotations-only, but modest.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single front-loaded sentence that enumerates the four value categories with no padding or repetition. Every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no input parameters, no output schema, and safety covered by annotations, the description's main job is to convey what the caller receives, which it does by naming the four reference sets. It would be stronger if it indicated the return structure or clarified the overlap with get_divisions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes zero parameters, so there is no parameter semantics to document; the baseline for a parameterless tool is 4. The description correctly implies no filtering input is required.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('Get') and resource ('reference data') and enumerates exactly which reference sets are returned (division, campus, delivery-mode, sorting). That is more informative than a tautology. However, it does not differentiate itself from the sibling get_divisions, which appears to overlap with the 'division' reference values it claims to return.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to call this tool versus alternatives, nor any mention of prerequisites or the overlapping get_divisions sibling. Usage must be inferred entirely from the noun phrase.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_divisionsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint and destructiveHint=false, so the safety profile is covered. The description adds only the semantic caveat about preferring these codes over campus abbreviations, and says nothing about return shape or volume. With annotations carrying the behavioral load, a 3 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single semicolon-joined sentence with the action first and the usage caveat second. No filler and nothing buried.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-argument, non-destructive lookup with no output schema, the description is nearly sufficient: it says what is listed and how to use the values. It could note that the result is a code/name mapping, but nothing essential to calling it correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes zero parameters, so there is nothing for the description to disambiguate and the baseline of 4 applies. No parameter behavior is misstated.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb+resource: it lists faculty/division codes. An agent can tell it returns a code list rather than records. It does not name or distinguish itself from the nearest sibling (get_reference_data), so it falls short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The clause 'use returned values rather than campus abbreviations' gives one concrete usage rule, implying this is the lookup to run before supplying a division code elsewhere. It names no alternative tool and gives no when-not-to-use condition, so guidance is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• retrieve_timetableA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint and destructiveHint=false, so safety and repeatability are covered. The description adds useful context about the state being previously saved and publicly shared (implying no mutation and no auth-scoped access), but says nothing about what is returned or error behavior for invalid ids.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence, front-loaded with the verb and scoped by the parameter, with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read tool with no output schema and annotations covering the safety profile, the description gives enough to invoke it correctly. A note on the source of the share id (e.g., returned by save_timetable) would make it fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% (only a pattern and minLength), so the description carries the burden; it explains that the single parameter is a 'public share id', giving semantic meaning beyond the bare format constraints. It omits where such an id comes from (presumably save_timetable).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Specific verb ('Load') plus resource ('previously saved Timetable Builder state') and an explicit identifying parameter ('public share id'). It is distinguishable from save_timetable/generate_timetable, though it does not explicitly contrast with get_current_sessions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'previously saved' implies the tool is for restoring an existing state when a share id is known, but there is no explicit when-to-use guidance, prerequisites, or named alternative such as get_current_sessions for live state.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_current_sessionsA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so safety behavior is covered. The description adds useful context beyond annotations: it filters to active sessions only and explains that session IDs come from non-header entries' values.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences with the core purpose front-loaded. The second sentence adds a concrete usage instruction without any filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 0-parameter read tool with no output schema, the description covers what the tool returns only partially. Saying to use non-header entries' values as session IDs gives some actionable guidance, but it does not describe the full return structure or entry format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero input parameters, so parameter semantics are not applicable and the baseline is 4. The description appropriately focuses on the output usage rather than inventing parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description gives a clear verb and resource: 'Get active academic sessions.' It scopes the result to active sessions, which distinguishes it from a general reference-data or search tool. However, it does not explicitly contrast itself with any sibling tool, so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied by the scope: call this to obtain active academic sessions. The second sentence adds an instruction for using the returned values as session IDs, but there is no explicit when-to-use or when-not-to-use guidance versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_coursesA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already establish read-only, idempotent, non-destructive behavior, so the description is free to add operational context — and it does: page numbering starts at one, page_size defaults to 20, sort direction defaults to asc, and critically that no automatic pagination is performed. That last point materially changes how an agent must loop, which is real value beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Tight and front-loaded: the primary action leads, then pagination mechanics, then data-source pointers, then parameter usage notes. Every sentence carries information, with only minor redundancy in the paging statements.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For an 11-parameter tool with zero schema documentation and no output schema, the description covers pagination and the two text filters well but leaves most reference-code filters opaque. Pointing to get_reference_data mitigates but does not close the gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% across 11 parameters, so the description must compensate. It documents page, page_size, course_code, course_title, and the sort default, and points to get_reference_data for filter codes, but campuses, sessions, divisions, delivery_modes, course_section_code, and search_course_description are never explained in either place.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource ('Search one page of courses') and immediately scopes the result set with the filter types (code/title and reference-code). The 'one page' framing plus filter list differentiates it from content-oriented siblings like search_course_titles and get_course_details, though it never explicitly names a sibling to contrast against.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives concrete routing guidance: session IDs come from get_current_sessions and other filter codes from get_reference_data, and it distinguishes exact course_code from keyword course_title. It lacks explicit when-not-to-use guidance, but the prerequisite sourcing is unusually actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• save_timetableA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds real context beyond the annotations: it discloses that the wrapper posts to the upstream ttb.utoronto.ca service, returns an upstream share id and a share link, and that only an anonymous share record is created. It does not mention that repeated saves create duplicate records, which the non-idempotent annotation hints at but the prose never confirms.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Front-loaded with the core action and return value, then the parameter requirement, then scope caveats. It is slightly wordy about the upstream URL mechanics, but every sentence contributes to correct invocation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description correctly documents the return shape (upstream share id plus an https://ttb.utoronto.ca/#!/?t= link). Annotations already cover the safety profile (write, non-idempotent, non-destructive), so remaining gaps are minor, e.g. behavior on repeated saves.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% and the single parameter is an unconstrained nested object (additionalProperties: true), so the description carries the burden. It does so well by specifying that "timetable" must be the TTB state object containing sessions, timetables, and plans, giving the agent the shape the empty schema omits.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: it stores a serialized Timetable Builder state and returns a public share URL. That verb (store/save) plus the share-URL return clearly separates it from siblings like generate_timetable and retrieve_timetable, which produce or fetch rather than persist.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clarifies one boundary explicitly ("creates an anonymous share record only; it does not enroll students"), which tells the agent this is not an enrollment action. However, it never says when to prefer this over siblings such as retrieve_timetable or what precondition the state must come from, so usage is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_course_detailsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly/idempotent/non-destructive, so the safety profile is covered. The description adds genuinely new context: the note that this endpoint has no session parameter, and the id-chaining hint toward generate_timetable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Front-loaded with the primary purpose, then example, filter, and downstream usage in short lines. The 'no session parameter in the supplied API reference' sentence is slightly meta but earns its place by preventing a wrong assumption.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, so the description's enumeration of returned fields (sections, meetings, rooms, instructors) is necessary and present. Coverage of error/empty-result behavior is absent but not critical for a read-only detail lookup.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must carry the load, and it does add value: a concrete course_code example (CSC108H1) and an explanation of the section_code filter. It does not spell out the pattern constraint (^[A-Za-z0-9]+$) that the schema enforces.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb (Get) and resource (course code's sections, meetings, rooms, instructors), and names the returned entities, which cleanly separates it from search_courses/search_course_titles lookups.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives an example input (CSC108H1), clarifies the optional F/S/Y filter, and points to the downstream tool (generate_timetable) that consumes the returned course id. It lacks explicit when-not-to-use guidance versus search_courses, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_timetableA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover the safety profile (readOnly, idempotent, non-destructive), and the description usefully reinforces that this is a planning-only operation that neither enrolls nor writes an ACORN timetable. It adds solver semantics and the one-plan-per-term grouping, but says nothing about how a result is shaped or any limits on plan size.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Roughly five sentences, each carrying distinct information: purpose first, then per-field meaning, then the negative scope caveat. No filler or restated title.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-argument tool with no output schema and clear annotations, the description covers purpose, inputs, and what it does not do. It is slightly thin on what the returned plans actually contain (sections, times, identifiers), which is the one thing only the description could supply.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With only one top-level parameter and low coverage in the reported schema descriptions, the description compensates well: it explains that each plan maps to one term, that activity_types are section types (Lecture/Tutorial/Practical), that preference is early/balanced/late, and that blocked_times use weekday names with 24-hour HH:MM values. Some of this restates schema enums/patterns, but the grouping and clock-format meaning go beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource: asks the university's solver for conflict-free lecture, tutorial and practical sections. It also carves out a clear boundary against siblings ('This does not enroll students or write an ACORN timetable'), which separates it from save_timetable and retrieve_timetable without opening their schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives an explicit input dependency ('course_id values come from get_course_details') and a negative scope statement ruling out enrollment/timetable writing. It stops short of naming save_timetable as the follow-up step for persisting a generated plan, so the alternative is implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.