Server Quality Checklist

Tool Scores

• search_transcriptsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations include readOnlyHint=true, so the tool's safety profile is already known. The description adds value by clarifying the scope of search (titles, participants, tags, speakers, spoken text). However, it does not disclose result format, ordering, or pagination behavior, leaving some behavioral aspects unexplained.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that directly states the tool's purpose and searchable fields. No wasted words, and the important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple with two parameters and a read-only annotation, but there is no output schema. The description does not mention what the search returns (e.g., matching transcript IDs, snippets), which leaves some ambiguity for an agent. For a search tool, clarifying the output would make it more complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers query with a case-insensitive description and limit with numeric constraints. The description adds semantic meaning to query by specifying the fields it applies to, but says nothing about limit's behavior. With schema description coverage at 50%, the description partially compensates but does not fully clarify both parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Search titles, participants, tags, speakers, and spoken transcript text.' It uses a specific verb (search) and resource (transcripts), and enumerates the searchable fields, which distinguishes it from sibling tools like list_transcripts and get_transcript.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for finding transcripts through search but does not explicitly state when to use this tool versus siblings or provide exclusions or alternative guidance. The verb 'Search' gives context, but no detailed usage instructions are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_transcriptA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true, indicating a safe read operation. The description adds the detail that it returns a 'complete' transcript, but does not disclose other behavioral aspects such as error handling or return format. This is useful but not extensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no redundant information. It is front-loaded and efficiently communicates the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple get-by-ID tool, the description is largely sufficient given the annotation and full parameter schema. However, the absence of an output schema means the description could have elaborated on what 'complete transcript' includes, leaving slight ambiguity about the return structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers the 'id' parameter completely with a description and example. The description does not add additional meaning beyond what the schema already provides, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get one complete transcript by its ID' uses a specific verb and resource, clearly indicating a single-resource retrieval operation. It distinguishes itself from sibling tools list_transcripts and search_transcripts by focusing on a specific ID-based fetch.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you have a transcript ID, but it does not explicitly state when to use this tool versus alternatives like search_transcripts or list_transcripts. No exclusions or alternative recommendations are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_transcriptsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With readOnlyHint=true annotation, the safety profile is already covered. The description adds that the tool returns metadata (not full content) and supports filtering, which is useful. However, it does not disclose behaviors such as pagination, ordering, or how the limit parameter behaves beyond the schema. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that conveys the core purpose and filtering capability without unnecessary words. It is well-structured and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only list operation, the description is adequate: it indicates the return type (metadata) and filter options. It does not detail pagination or the exact shape of returned metadata, but given the simple nature and lack of output schema, the description provides sufficient context for use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 67% (tag and participant have descriptions, limit lacks one). The description reinforces the role of tag and participant as filters but does not mention limit. It adds a high-level framing but does not significantly enhance the schema's own parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and resource ('transcript metadata'), distinguishing it from siblings: get_transcript likely retrieves a single full transcript, and search_transcripts implies broader search. The optional filter phrase adds specificity without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for browsing metadata with optional filters, but it does not explicitly state when to prefer this over search_transcripts or get_transcript, nor does it provide exclusions or alternative recommendations. The sibling names offer context, but the description itself lacks direct usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.