Server Quality Checklist

Tool Scores

• pingA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. 'Health check' implies a non-mutating, low-risk operation, but it does not explicitly state that there are no side effects, what exactly is checked, or any authentication expectations. The output schema handles return-shape details, so this minimal disclosure is acceptable but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two words and contains no filler or repetition. It is front-loaded and appropriately sized for a simple, parameterless health-check tool, though it could be slightly more structured with a full sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters, an output schema, and low complexity, the description is largely sufficient. Still, it does not explicitly address when to call this tool, what a successful or failed health check means, or whether any setup is required, so there is a small but real gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is nothing to document. Per the baseline for zero-parameter tools, the description does not need to add parameter-level meaning, and none is missing.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Health check' clearly conveys that this tool is a liveness or connectivity probe, even though it lacks a formal verb. It is naturally distinct from the sibling content-retrieval tools, so an agent can infer its role without opening the schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The purpose implies the usage context: call this tool to verify that the service/API is reachable or healthy. However, there is no explicit statement of when to prefer it over alternatives or any exclusion criteria, leaving some inference to the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_lessonsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden and does well: it discloses grouping by module, playback ordering, the aggregate duration field, and a concrete pitfall — pass the slug, not the numeric video_id — when calling get_lesson_transcript. It does not cover error behavior or state read-only status in so many words, but the verb 'list' and the read-style tool family make side effects implausible.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences with zero waste: purpose, parameter guidance, return contract, and a downstream warning each earn their place. The return-structure detail is justified because no output schema exists, and the slug-vs-video_id warning prevents a realistic invocation error.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read tool with no annotations and no output schema, the description covers the input source and format, the full return dict shape, ordering, grouping, and downstream routing. Minor gaps remain — the possible values of the lesson 'type' field and behavior on an invalid course_slug — but nothing an agent needs to make the call correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%; the schema only labels the property 'Course Slug.' The description compensates by giving the parameter's source ('from list_enrolled_courses') and a concrete format example ('agentic-ai'), which is exactly the semantics an agent needs to fill the argument correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence names a specific verb and resource — 'List lessons for one course' — and adds a distinguishing scope: 'grouped by module in playback order.' This clearly differentiates it from sibling list_enrolled_courses, which operates at the course level, and from the get_* lesson tools that fetch individual artifacts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The Args section anchors the tool in a workflow: 'course_slug: from list_enrolled_courses, e.g. agentic-ai' implies the prerequisite call, and the closing sentence routes the agent forward to get_lesson_transcript with the correct identifier. However, it never explicitly states when not to use this tool or contrasts it with siblings like get_lesson_frame or get_lesson_slides.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_enrolled_coursesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It clearly states that specializations are flattened into constituent courses, and enumerates the fields in each item (slug, name, type, progress_percent, parent info). This gives the agent a good sense of the output structure and the flattening behavior. However, it does not mention whether there are any limits (e.g., number of courses) or whether the list is sorted or filtered; these minor gaps prevent a perfect score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: the opening sentence states the purpose, the second sentence explains a key behavior (flattening), the third lists the output fields, and the final sentence gives a usage tip. Every sentence adds value, and there is no redundancy or fluff. It is front-loaded with the core purpose, making it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a no-parameter list tool with an output schema, the description covers all essential aspects: what it does, how the data is shaped, and how to use the results with other tools. The output schema likely formalizes the structure, so the description adds narrative context without repeating schema details. No critical information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema is trivially complete (coverage 100%). The baseline for zero parameters is 4, and the description correctly adds no parameter-specific details because there are none. It does mention the output fields, which aids in interpreting what the tool returns but does not affect parameter semantics. A score of 4 reflects the baseline appropriateness.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a precise verb and resource: 'List every course you're enrolled in on learn.deeplearning.ai.' It clearly defines the scope (courses, not lessons) and even details the flattening of specializations, making it unambiguous. While it doesn't name sibling tools explicitly, the distinct purpose is evident and no confusion arises among the listed siblings (list_lessons, get_lesson_transcript, etc.).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives practical usage guidance by instructing the agent to use the returned `slug` for subsequent calls to `list_lessons` or `get_lesson_transcript`. This effectively communicates how the output feeds into other tools. It does not explicitly state when to avoid this tool, but since it is the only course-listing tool among siblings, that is not a critical omission. The guidance is clear and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_lesson_slidesA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does so thoroughly. It discloses return structure, side effects (writing JPEGs to save_dir), parameter effects (threshold, min_gap_seconds dedup), and the ffmpeg dependency. It also explains include_images behavior for token-conscious use.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Though lengthy, the structure is efficient: a one-sentence summary, an Args list with line-per-parameter detail, and a crisp description of return format and dependencies. Every sentence earns its place, and the most important 'auto-detect' behavior is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for a tool with no output schema and no annotations. It explains the return order (JSON summary then caption+image), how to disable images, file side effects, and the external ffmpeg requirement. Nothing essential for calling the tool correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains every parameter, including tuning guidance for threshold, what max_frames caps, how min_gap_seconds suppresses duplicates, filename pattern for save_dir, and the purpose of include_images. This exceeds what the schema alone provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening sentence, 'Auto-detect slide changes in a lesson video and return one frame per slide,' states a specific verb and resource. It clearly differentiates from sibling tools like get_lesson_frame by focusing on automatic detection across an entire lesson rather than a single frame.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies its use case ('Auto-detect slide changes') and references list_enrolled_courses and list_lessons for slugs, but it never explicitly states when to use this tool versus get_lesson_frame or get_lesson_transcript. There are no exclusions or alternative routing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_lesson_transcriptA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full transparency burden. It discloses the error behavior for non-video lessons, explains the format parameter's two modes, and states the exact return shape. It does not explicitly mention whether the operation is read-only, but the verb 'fetch' strongly implies it, and no hidden side effects are suggested.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: a one-sentence purpose, a concise args list with practical guidance, and a return/error note. Every sentence adds useful information and nothing is redundant.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description covers all essential aspects: what it does, how to populate each parameter, what the return object contains, and when it will fail. No critical information needed to call it correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must fully compensate. It explains the provenance of course_slug and lesson_slug, gives an example format for lesson_slug, and details the two enum values for format with behavioral descriptions. This goes far beyond the bare string/enum schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Fetch the full transcript for one video lesson,' which is a specific verb and resource. It clearly distinguishes itself from sibling tools like get_lesson_frame and get_lesson_slides by focusing solely on transcripts, and further clarifies that only video lessons are supported.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use this tool: for video lesson transcripts. It explicitly warns that 'other lesson types raise an error,' giving a when-not condition. It also specifies the source of each slug (from list_enrolled_courses and list_lessons), though it does not explicitly name alternative tools for non-video content.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_lesson_frameA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral disclosure burden. It covers the write side effect to save_path (including ~ and $VAR expansion, parent dir creation, and absolute path in response), the conditional inline image via include_image, the ffmpeg dependency, and the return shape involving JSON summary, caption, and inline image.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The one-sentence summary is front-loaded, followed by a compact Args block where each line adds a distinct detail. No sentence is wasted; the return behavior and external dependency are stated in two short sentences at the end.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema and no annotations, the description is complete for a 6-parameter tool: it documents every parameter, prerequisites from sibling tools, return format, optional image inclusion, file path behavior, and the ffmpeg requirement. An agent has enough information to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must fully explain every parameter, and it does. Each of the six parameters gets meaningful guidance: provenance for slugs, timestamp meaning, quality resolution/speed, save_path side effects, and include_image behavior including its default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Screenshot a single video frame from a lesson at the given timestamp.' This clearly distinguishes the tool from sibling tools such as transcripts and slides, which serve different resources and purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear usage context: which slugs come from which sibling tools, the quality/speed tradeoff, and the save_path/include_image usage pattern. It does not explicitly state when to prefer this over get_lesson_transcript or get_lesson_slides, but the purpose and dependencies are clear enough to guide selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.