Server Quality Checklist

Tool Scores

• list_exercisesC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations available, the description carries the full burden of behavioral disclosure. It only mentions 'brief' but fails to describe pagination, ordering, side effects, or other key behavior like whether results are filtered by default or if all filters are optional.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (one sentence) and front-loads the purpose. It efficiently lists filter categories. However, it could benefit from clearer formatting or distinct sections for each filter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having 6 parameters, no output schema details are provided in the input, and the description is too brief to fully equip an agent. Key details like 'brief' meaning, default limit behavior, and whether results are sorted are missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, yet the description only lists filter names without explaining their types, formats, or expected values. For example, 'topic substring' is ambiguous: it does not specify if it's case-sensitive or supports wildcards. No examples are given.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists exercises and enumerates the available filters (scheda, topic, type, section, status). However, it does not explicitly differentiate from sibling tools like get_toc or get_exercise, which are distinct in purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, scenarios, or conditions that would make this tool preferable over siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_exerciseC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It lists what is returned but does not disclose behavioral traits such as read-only nature, performance implications, or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence listing key components. It is concise and front-loaded with 'Full exercise', though it could be more structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the existence of an output schema, the description adequately covers the tool's purpose and return content. It is sufficient for basic understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, yet the description does not mention the exercise_id parameter or its meaning. Thus it adds no value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies that it returns the full exercise including instruction, content, metadata, source page, and history. It is clear about the resource, but does not differentiate from siblings like get_page or list_exercises.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. It does not mention prerequisites, caveats, or when to avoid using it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_pageA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries burden. States 'full content' but doesn't specify what's included (e.g., images, formatting). Lacks disclosure of read-only nature, but for a simple retrieval it's adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences, front-loaded purpose, no fluff. Efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given low complexity, presence of output schema, and clear purpose/parameters, description is sufficiently complete for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage 0%, description adds meaning by mapping pdf_page and book_page to lookup methods. However, does not clarify exclusivity or precedence if both provided. Adds some value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb ('get') and resource ('one book page'), specific content (OCR markdown), and distinguishes from siblings like get_toc, list_exercises, get_exercise.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Clearly states look up by PDF page or book page, but does not explicitly mention when not to use (e.g., for exercises use get_exercise). Context is clear though.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_tocA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the output structure in detail (3 levels with content descriptions). It does not mention read-only or mutation behavior, but given the nature of a TOC retrieval, the behavioral traits are mostly inferred.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that efficiently conveys the structure using arrows and specific details. Every word adds value, and the key information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and the presence of an output schema, the description explains the output structure sufficiently. It covers the three levels and their contents, though it could slightly benefit from mentioning usage context or response format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the description need not add parameter info. The baseline for 0 parameters is 4, and no additional semantics are required.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool provides a table of contents with 3 levels (topic areas, schede, sections), specifying details like page ranges, exercise counts, completion checkmark, and kind. It distinguishes from siblings (get_page, list_exercises, get_exercise) by focusing on hierarchical structure.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for navigating the structure before fetching specific pages or exercises, but does not explicitly state when to use this tool versus alternatives. No when-not or prerequisite guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.