Server Quality Checklist

Tool Scores

• course_detailA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It describes the return content but does not disclose potential failure modes, side effects, or behavior for invalid inputs. The 'Get' verb implies read-only, and parameter defaults are explained, but edge cases are not addressed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description starts with a concise summary, then a compact Args block that provides exactly the needed parameter guidance. No superfluous information, well-structured and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists, the description covers purpose and parameters well. It might benefit from noting fallback to current term/year or what happens if the course is not found, but overall it is adequate for the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no descriptions, and the description thoroughly explains each parameter with examples, allowed values for semester, and default behavior for empty year/semester. This fully compensates for the 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: retrieving full details for a specific course, listing the types of information included. This distinguishes it from siblings like 'search' and 'batch_course_details'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for single-course detail lookups but does not explicitly contrast with batch_course_details or search. It provides clear context for when to use it but lacks explicit exclusions or alternative tool references.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden for behavioral disclosure. It adds useful context about defaults (e.g., 'Empty = current term' and 'Empty = current year') and specifies the shape of the output (list of codes, titles, instructors, meeting times, seat availability). However, it does not mention limits, pagination, or how filters combine, which would improve transparency further.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a two-sentence purpose statement followed by a bullet-like 'Args:' list. Every sentence provides necessary information without waste. It is front-loaded with the tool's purpose and output, making it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 optional parameters and an output schema, the description is largely complete: all parameters are documented with examples and defaults, and the return type is summarized. The main gap is that the description does not explain how multiple filters interact, sorting, or result limits, but the presence of an output schema and the straightforward search nature keep it sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only provides titles and defaults, with 0% description coverage. The tool description compensates fully by explaining each parameter with examples (e.g., 'keyword: Free-text search, e.g. "linear algebra", "MATH 1530"') and clarifying default behavior for semester and year. This adds significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search Brown's course catalog (Courses@Brown / CAB)' and defines the output as 'a compact list of matching courses with codes, titles, instructors, meeting times, and seat availability.' This distinguishes it from siblings like course_detail (which would presumably provide full details) by emphasizing the compact list format.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides guidance on what each parameter does (e.g., 'keyword: Free-text search... department: Department code...') and implies use for searching courses. However, it does not explicitly state when to use this tool versus its siblings (departments, course_detail, batch_course_details, schedule_check) or exclude any scenarios, so usage guidance is primarily implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• schedule_checkA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the behavioral disclosure burden. It conveys a non-mutating 'check' action, implying a read-only operation, but does not mention any potential side effects, authentication requirements, rate limits, or behavior on invalid inputs. This is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a single-sentence purpose followed by a clearly formatted Args list. Every sentence provides necessary information without fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool of moderate complexity (multiple courses, semester/year parameters), the description, combined with the presence of an output schema, covers the essential aspects. There is no obvious missing information that would prevent an agent from using the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no parameter descriptions (0% coverage), but the description fully compensates by explaining each parameter: 'codes' with examples, 'semester' with allowed values and default behavior, and 'year' with format and default. This adds significant meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: 'Check meeting times for multiple courses to spot conflicts.' This distinguishes it from siblings like course_detail, which focuses on single course details, and batch_course_details, which likely returns course details without conflict analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description establishes clear context: use when you need to analyze meeting times across multiple courses to identify conflicts. However, it does not explicitly mention when not to use it or name alternative tools, so it stops short of a full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• batch_course_detailsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the burden. It discloses that the tool performs parallel fetches and is faster. It also clarifies default behavior for empty semester and year parameters. It does not mention error handling or limits, but those are less critical given the output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is succinct and well-structured: an intro sentence, a comparative sentence, and a clear Args block. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's purpose, usage context, parameters, and performance benefit. The output schema handles return format. It could mention potential limitations like maximum codes, but for a batch fetch tool this is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no descriptions, but the description's Args section adds critical meaning: it provides example codes, enumerates valid semester values, and explains that empty strings default to the current term/year. This fully compensates for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get details for multiple courses at once (parallel fetch).' It uses a specific verb and resource, and distinguishes itself from sibling course_detail by emphasizing batch operation and speed.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly mentions the alternative 'calling course_detail repeatedly' and positions this tool as faster for batch operations. However, it does not explicitly state when not to use it (e.g., for single course requests), though it is implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• departmentsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It discloses that the list is not exhaustive, setting expectations about coverage, and provides context on its use. While it does not detail output format, the presence of an output schema covers that. The added limitation note goes beyond a bare statement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main purpose and followed by a necessary limitation. There is zero wasted wording; each sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple parameterless tool with an output schema, the description fully covers what the agent needs to know: what it lists, that it is for use with search, and that it is non-exhaustive with a fallback. The presence of the output schema removes the need to describe return values, so the description is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description does not need to explain parameters, but it does add context about how the output (department codes) integrates with the search tool, which is meaningful for the tool's use.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists common Brown department codes and explicitly ties this to use with the search tool. This distinguishes it from sibling tools like search or course_detail, making the purpose unambiguous and specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance: use it to obtain department codes for the search tool, and if the department is not listed, use keyword search instead. This clearly states when to use this tool and provides an alternative, satisfying the when/when-not requirement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.