Server Quality Checklist

Tool Scores

• search_knowledge_baseC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It mentions 'local' files, adding a scope constraint, but does not disclose whether the operation is read-only, how results are sorted, or any side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a concise single sentence with no waste, making it easy to parse. However, it is slightly under-specified for the tool's functionality, preventing a perfect score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (two parameters, output schema present), a short description is acceptable, but it omits details about result ordering, limits, and the distinction between searching local vs. other sources. The lack of annotations further reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero description coverage, and the description only adds 'keyword' as a hint for the query parameter. The 'limit' parameter is entirely unexplained, so the description fails to compensate for the schema's lack of detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Search', the resource 'local knowledge files', and the method 'by keyword'. It distinguishes from siblings like get_document by indicating a search operation, but does not explicitly contrast with alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus siblings such as get_document or build_rag_prompt. No use cases, exclusions, or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• normalize_user_queryC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. However, it fails to specify what 'normalize' entails (e.g., casing, trimming, stemming), whether it modifies the input or returns a new value, or any side effects. The 'lowercase' parameter is not mentioned, leaving significant ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no unnecessary words. It is front-loaded with the verb and resource, making it easy to scan. However, the extreme brevity borders on under-specification, though that is more a completeness issue than a conciseness one.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite the tool being relatively simple, the description is too minimal. It lacks behavioral details, parameter explanations, and any firm usage heuristics. Given the absence of annotations and the presence of an output schema, the description still fails to provide a complete picture of when and how to use the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description adds no information about the 'query' or 'lowercase' parameters. The parameter names offer some hints, but the description does not explain their meaning, types, or usage. This provides no added value beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Normalize') and the resource ('user text'), and provides context for when it is performed ('before retrieval, routing, or agent planning'). It is distinct from sibling tools like search_knowledge_base or get_document, though it does not explicitly differentiate itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'before retrieval, routing, or agent planning' implies the usage context, but there is no explicit guidance on when not to use the tool or which alternatives might be more appropriate. The guidance is implied rather than directive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• build_rag_promptB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full behavioral disclosure burden. It only states the tool 'builds' a prompt without explaining how the question and context are combined, what operational meaning 'grounded' has, or whether this is a pure transformation with no side effects. This is a purpose statement rather than a behavioral specification.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no filler or redundancy. It is front-loaded and efficient, though the minimalism contributes to gaps in other dimensions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although an output schema exists, the description still lacks guidance on prerequisites, when to use this tool versus siblings, and how answer_style alters the output. For an agent making a selection decision, the description is not sufficiently complete to ensure correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the lack of parameter documentation. It only paraphrases 'question' and 'context' without adding format, constraints, examples, or semantics, and it entirely omits the answer_style parameter beyond what the schema's enum already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action (build) and the resource (a grounded RAG prompt) from question and retrieved context, which distinguishes it from the sibling search/retrieval tools. It doesn't fully elaborate on what a 'grounded RAG prompt' entails, but the core purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'retrieved context' implies this tool should be used after retrieval tools like search_knowledge_base or get_document, but there is no explicit when-to-use vs. when-not-to-use guidance. It also doesn't mention relationships to normalize_user_query or how answer_style affects invocation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_documentB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavior. It indicates a read-only operation with no side effects, but does not mention error handling (e.g., missing file), path restrictions, or access constraints. The output schema covers return values, but behavioral context is limited.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the primary verb and resource. It contains no redundant information and is appropriately sized for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one parameter and an output schema, but the description lacks usage guidance and behavioral details that would help an agent select and invoke it correctly. It is adequate for a basic read operation but leaves gaps around error cases and when to prefer this over sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter meaning. The description implies 'relative_path' is relative to the configured knowledge directory but does not explicitly explain the parameter, its format, or any constraints. This is minimal value beyond the parameter name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Read') and resource ('a document from the configured knowledge directory'). It distinguishes from siblings like search_knowledge_base by indicating a direct file read by path rather than a search or query operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as search_knowledge_base. The description does not mention prerequisites, limitations, or preferred scenarios, leaving the agent without explicit usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• health_checkA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing behavior. It states that the tool 'returns' data, implying a read-only operation, but it does not explicitly confirm safety, error behavior, or potential side effects. For a health check with no parameters, this is adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no redundant words or filler. It effectively communicates the tool's purpose in minimal space.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (zero parameters) and the presence of an output schema, the description's high-level overview of what is returned ('server health and runtime configuration') is sufficient. No additional context about return values is needed since the output schema is available.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and schema coverage is 100% (vacuously). The description adds no parameter-specific details, but none are needed. The baseline for zero parameters is 4, and the description appropriately avoids inventing unnecessary parameter information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Return') and clearly states both the resource ('server health') and additional scope ('runtime configuration'). It is unambiguous and fully distinguishes this tool from sibling tools, which are knowledge-base oriented.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use this tool versus alternatives, nor does it provide exclusions or prerequisites. The intended use is implied by the name and description—verifying server health—but no explicit guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.