Server Quality Checklist

Tool Scores

• list-sourcesA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are empty, so the description carries the full burden. It states the tool lists sources with capabilities, implying a read operation, but does not disclose whether it requires permissions, has pagination, or any side effects. For a simple list, this is minimally adequate but leaves gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is a single concise sentence with no fluff. Every word serves a purpose, and it is front-loaded with the main action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one optional parameter and no output schema. The description mentions 'capabilities' but does not specify the return format or any details about the list (e.g., pagination, sort order). It is adequate but could be more complete for an agent to fully understand the response.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the description only repeats the optional filter capability already captured in the schema's parameter description. It adds no new meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool lists all available knowledge sources with their capabilities, and optionally filters by namespace. The verb 'List' and the resource 'knowledge sources' are specific, and the optional filter distinguishes it from sibling tools like search-knowledge (which likely searches within sources) and get-source-schema (which retrieves schema for a specific source).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus siblings. For example, it does not explain when to use list-sources instead of search-knowledge or get-source-schema. The description only mentions the optional namespace filter, but no contextual cues for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-source-schemaA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With empty annotations, the description carries burden. It accurately describes the tool as retrieval ('Get'), but does not disclose any potential side effects, error behaviors (e.g., if source does not exist), or auth requirements, which limits transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 18 words, front-loading the purpose. Every word is necessary, and there is no redundant or vague language.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no output schema), the description is sufficiently complete: it explains the tool's action and the parameter. However, it could explicitly mention the return format (e.g., 'returns schema and capabilities JSON'), but overall it covers the essential context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already fully documents the parameter. The description adds no additional meaning beyond what is in the schema, earning a baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'schema and capabilities for a specific knowledge source', and it distinguishes itself from sibling tools 'search-knowledge' and 'list-sources' by focusing on a specific source's schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need schema for a specific source, but it does not provide explicit guidance on when to use this tool versus alternatives like 'search-knowledge' or 'list-sources', nor does it mention conditions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search-knowledgeA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries the burden. It reveals that the server determines the retrieval strategy internally, but does not disclose whether the operation is read-only, side effects, authentication needs, or rate limits. This is adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the main purpose, followed by targeted parameter guidance. No unnecessary words, every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers basic usage and parameter hints, but lacks details on return format, pagination, or result structure. With no output schema, the description could provide more context about what results look like.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. Description adds context on filters and search_type beyond schema descriptions, reinforcing their purpose. However, it does not add significant new details for other parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it performs a unified search across all knowledge sources, with a specific verb and resource. It distinguishes from siblings like list-sources and get-source-schema by focusing on searching rather than listing or schema retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides guidance on using filters and search_type parameters, and notes that the server determines the best strategy. However, it does not explicitly state when to use this tool versus alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.