Server Quality Checklist

Tool Scores

• get_kb_articleA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Even though readOnlyHint is already set, the description adds concrete behavioral details: no side effects, a header line with metadata fields, HTML-to-plain-text conversion, and truncation with a marker when cut. This gives the agent an accurate mental model of the response without contradicting annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well organized into purpose, usage guidance, and behavior, with no wasted sentences. It front-loads the core function and places important usage and truncation details where they are easy to find.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter read-only tool, the description is complete: it names the input forms, explains when to use it, describes the output format, and warns about truncation. The presence of an output schema further reduces the need for the description to detail return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents both the article parameter (number or URL) and max_chars (range, default, truncation-marker hint). The description reinforces these details in context but adds little beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states a specific verb ('Fetch') and resource ('full text of one AVEVA knowledge base article') and lists the article sections it covers. It clearly differentiates from the sibling tools by focusing on full-text retrieval rather than search or product listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs when to use the tool: after search_pi_kb when an excerpt looks relevant, or directly when a user supplies an article number or support link. It also explains why reading the full article is necessary, because search excerpts are truncated and routinely omit the fix.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_pi_kbA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond readOnlyHint, the description discloses no side effects, English/KB scope, exclusions (product docs, community posts, product news), and that missing auth returns a login instruction instead of failing. It also pre-announces the result shape: match count then entries with article number.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but every sentence earns its place: purpose, use cases, exclusions, auth behavior, and return shape. Key facts are front-loaded and nothing extraneous is included.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with three params and two siblings, the description covers all decision-relevant context: when to use it, scope restrictions, auth prerequisite, output format, and how it relates to get_kb_article and list_kb_products. An output schema exists, so return details are also backed by structured info.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline applies. The input schema already gives detailed meanings, examples, defaults, and ranges for query, products, and n_results. The description adds a small bit of context by saying n_results limits returned entries, but doesn't need to compensate for any schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Search') and resource (AVEVA Customer Support knowledge base) and lists the exact return fields: article numbers, titles, products and excerpts. It also differentiates from siblings by noting the article number is what get_kb_article needs and referencing list_kb_products.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says when to use this tool (support-desk material, error messages, known issues, workarounds, how-to notes) and when not to use it (conceptual or reference questions should go to product documentation). It also covers the signed-in session requirement and the fallback behavior.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_kb_productsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint=true and openWorldHint=true, and the description reinforces these with 'Read-only; no side effects'. Beyond the annotations, the description discloses critical behavioral traits: the non-obvious default (no argument returns only PI System products, not everything), the caching behavior for process lifetime, the return format (one name per line, alphabetical), and the 'note if nothing matched' outcome. The scale context ('over a thousand product names') explains why this default exists. All of this is additive value that annotations cannot express.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences in a purposeful order: purpose, usage guidance with concrete examples, behavioral caveats, return format. Every sentence carries distinct information with zero filler. The slight redundancy of 'Read-only; no side effects' against the annotation is acceptable reinforcement rather than waste. Front-loaded with the core purpose, which is exactly what an agent scanning tool lists needs.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one optional parameter, an output schema, and strong annotations, the description is complete. The most dangerous gotcha — that omitting the argument does not mean 'all products' but rather 'PI System products only' — is explicitly highlighted. The description covers when to use, when not to use, default behavior, caching, output format, and empty-result behavior. Nothing an agent would need to call this correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the schema description is already strong with worked examples ('vision', 'historian', 'adapter'). The description adds one useful clarification beyond the schema: passing `contains` expands beyond the default PI System set ('find products outside that set'), which resolves an ambiguity in the schema's phrasing 'Omit to list PI System products only' — one might otherwise infer contains merely filters within that set. That semantic disambiguation is real value, though the schema does most of the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence states a specific verb ('List'), a specific resource ('AVEVA product names'), and ties it precisely to the products argument of the sibling tool search_pi_kb. It is immediately distinguishable from both siblings: search_pi_kb (which searches articles) and get_kb_article (which retrieves a single article). This is textbook purpose clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives two explicit trigger scenarios (narrowing cross-product search results by exact spelling, and resolving informal user names like 'PI AF' to AVEVA canonical form) and an explicit exclusion ('Not needed for a first, unscoped search'). It also names the sibling tool whose results this tool supports, giving the agent a clear decision rule for when to call it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.