Server Quality Checklist

Tool Scores

• search_sourcesA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It states the search operation and scope but does not mention return format, pagination, side effects (though likely read-only), or any constraints. This lack of detail beyond the basic intent leaves significant ambiguity for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: one sentence defining the tool's action and scope, followed by a two-argument listing. All information is front-loaded and every word contributes value, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-parameter search tool with an output schema, the description covers the essential scope and parameters. However, it lacks comparison to sibling tools and does not clarify edge cases like empty results or limit behavior. Still, given the low complexity and presence of an output schema, it is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, but the description compensates by explaining both parameters: query is a search term for title, channel, or URL, and limit is the maximum result count. This adds meaningful semantic context beyond the schema's raw types and defaults, though it could be more detailed (e.g., query matching behavior).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Search') and a defined resource scope ('ingested sources, playlists, and videos'), distinguishing it from sibling tools like search_knowledge which target knowledge items. It also mentions searchable attributes (title, URL, metadata), making the purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (when searching ingested media sources), but it does not explicitly say when to prefer it over alternatives like search_knowledge or get_source. No exclusions or comparative guidance is provided, leaving the usage context somewhat implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_evidenceA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. The verb 'Retrieve' implies a safe read operation, and listing the returned components adds context. However, it does not explicitly confirm read-only behavior, error scenarios, or any side effects, leaving some transparency gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: a single clear sentence followed by a brief Args section. Every word earns its place, and there is no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter get-by-ID tool, the description covers purpose, the parameter, and the nature of the returned data. An output schema exists to fully define return structure. Missing only explicit usage guidance, which prevents a perfect score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only declares evidence_id as a string with no description. The tool description adds that it is a 'UUID string of the evidence record', giving the parameter real semantic meaning beyond type and name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Retrieve' followed by a precise list of returned content (raw source evidence snippet, timestamps, video association, confidence). This clearly distinguishes it from sibling tools like get_source and get_knowledge, which presumably retrieve different resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: this tool fetches a specific evidence record by its ID. However, it provides no explicit guidance on when to use this tool versus alternatives (e.g., search_sources or get_source), nor any exclusions or prerequisites beyond the required evidence_id.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_sourceA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the transparency burden. It discloses that the tool returns details, ingestion status, and published package versions, but omits error behavior, permissions, or read-only confirmation. Still, it provides substantive information about the returned data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with a clear Arg spec, no fluff, and front-loads the primary action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (single parameter, no nested objects) and has an output schema, so the description need not detail returns. It covers the entity types and mentions key data facets ('ingestion status', 'published package versions'). It lacks explicit guidance for handling invalid IDs but is otherwise complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only defines source_id as a string. The description adds that it is a UUID and can represent a source, playlist, or video, substantially enriching the semantic meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Get' and identifies the resource as source/video/playlist, clearly distinguishing it from sibling tools like search_sources and get_knowledge by type and operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when an ID is available but does not explicitly state when to prefer this over search_sources or mention exclusions. No alternative tools are referenced.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_knowledgeA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of disclosing behavior. It mentions 'published' and 'hybrid semantic + lexical retrieval,' which provides some context, but it does not explicitly state that the operation is read-only or describe potential side effects, pagination, or other operational details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured, beginning with a clear one-sentence summary followed by concise parameter explanations. It is slightly longer than necessary but each line adds value, making it efficient and readable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's purpose, parameter semantics, and retrieval method, and an output schema exists so return values need not be explained. It lacks explicit alternative usage but is otherwise sufficiently complete for a search tool with a straightforward scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides only titles and types (e.g., 'Query', 'Limit'), while the description adds substantial meaning by explaining each parameter, such as 'query: Natural language query to search knowledge items for' and 'min_confidence: Minimum confidence score filter (0.0 to 1.0).' This effectively compensates for the 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: 'Search published knowledge items and packages using hybrid semantic + lexical retrieval.' This distinguishes it from sibling tools like search_sources, which focuses on sources rather than knowledge items.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for searching knowledge items but does not explicitly state when to use it over alternatives or provide exclusion scenarios. There is no mention of when to prefer search_sources or other siblings, so usage guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_knowledgeA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It clearly states that this is a fetch operation and specifies what is included in the result ('bound evidence citations and relationships'). It does not mention error cases or permissions, but for a simple read operation the description provides meaningful behavioral context beyond the tool name.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two lines with a short Args block. It is concise, front-loaded with the primary action, and contains no filler. Every sentence adds useful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple get-by-UUID tool, the description is complete: it states the resource, the identifier format, and the expected result contents. An output schema exists, so the absence of detailed return formatting is acceptable. The description sufficiently equips an agent to select and invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides only a property name 'knowledge_id' with no description, so the description must compensate. The Args section explicitly explains that knowledge_id is a 'UUID string of the knowledge item', adding precise type and semantic meaning. This fully clarifies the parameter even though the schema coverage is 0%.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with 'Fetch a specific knowledge item by its UUID', using a specific verb and resource. It also identifies the returned content ('bound evidence citations and relationships'), which distinguishes it from sibling tools like get_evidence or search_knowledge.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes the usage context clear: use this when you have a specific knowledge item's UUID and need its details, evidence citations, and relationships. It does not explicitly mention alternatives or when not to use it, but the 'by UUID' scope implies a direct lookup rather than a search, which is a clear context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.