Server Quality Checklist

Tool Scores

• get_trackA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that the tool returns a track with its full lesson list, making the read-only nature clear. However, it lacks additional behavioral context such as error cases, permissions, or any restrictions, leaving some expectations implicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence that efficiently states the action, resource, and purpose without unnecessary detail. Every word contributes to understanding the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite lacking an output schema, the description explains what the returned data includes (track + full lesson list) and provides a concrete reason to use the tool. This is sufficient for an agent to decide when to call it and what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents the single required parameter 'slug' with a clear example ('rag'). The description adds no extra parameter meaning, but since schema coverage is 100%, the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves one track with its full lesson list, distinguishing it from list_tracks (listing all tracks) and read_lesson (reading a single lesson). The verb 'Get' plus the resource 'track' makes the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear use case: to see the structure of a course and pick the right lesson to read. It does not explicitly mention when not to use it or name alternatives, but the context of siblings and the stated purpose imply when it is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_lessonA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior, and it does mention the return format ('readable markdown-ish text with headings and code blocks'). However, it omits error behavior, such as what happens if the URL is invalid or not found, leaving a gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main purpose, and contains no filler. Every word contributes to understanding the tool's function and output.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with one parameter and no output schema, the description covers the core purpose, input, and output format. It is complete enough for an agent to use correctly, though it could briefly mention error handling for a fully comprehensive picture.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes the 'url' parameter, but the description adds that the URL must be 'as returned by search_lessons or get_track', clarifying its provenance and format. This extra context goes beyond the schema's basic field description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Fetch the full text of a lesson') and the required input (URL from search_lessons or get_track). This distinguishes it from sibling tools that search or list tracks, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a prerequisite: the URL must come from search_lessons or get_track. This gives clear context on how to obtain the URL, though it does not explicitly state when not to use the tool or mention alternatives for finding lessons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_lessonsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It communicates that this is a read-only search operation by stating it returns matching lesson titles and URLs, and it scopes the corpus (500+ tracks, free lessons). It does not explicitly state 'no side effects' or edge-case behavior, but the wording implies a safe, non-mutating lookup.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no wasted words. The first sentence front-loads the purpose and output; the second provides concrete usage guidance and a natural handoff to read_lesson.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's scope, return format, and when to use it, which is sufficient for a simple search tool. It does not describe result count/ordering, but the limit parameter is already in the schema and no output schema is present, so this is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with query and limit already described. The description does not add extra meaning beyond the schema; it only restates that it returns matching results. This matches the baseline of 3 for high schema-description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches the AI School curriculum and returns matching lesson titles and URLs. The verb 'Search' plus the resource scope distinguishes it from sibling tools like list_tracks, get_track, and read_lesson.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says to use this tool before answering an AI-topic question when a concrete, citable lesson would help, and that read_lesson should follow to get the text. It lacks explicit when-not-to-use or alternative exclusions, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tracksA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It discloses the substring filtering behavior on title or slug, which is not fully in the schema. It also clarifies the nature of tracks (structured courses on one subject). It does not mention pagination or output format, but this is a simple read-only list operation, and the core behavior is well covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main purpose, followed by a brief explanation and a usage hint. Every sentence earns its place with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with no output schema and no annotations, the description provides the essential context: what the tool lists, how filtering works, what a track is, and when to use it. It does not detail return fields, but for a list endpoint that is commonly understood. The description is complete enough for effective tool selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds meaning to the 'filter' parameter by specifying it matches substring of title or slug, which the schema only calls 'Optional substring filter.' This extra semantic context improves parameter understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List...course tracks') and specifies the resource ('AI School course tracks'). It distinguishes from siblings by noting that tracks are structured courses and by positioning this as the discovery step before drilling into lessons, which separates it from get_track and read_lesson.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit usage context: 'Use this to find the right course before drilling into its lessons.' This implies when to use it relative to lesson-related tools, though it doesn't explicitly state alternatives or when not to use it. The guidance is clear and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.