Server Quality Checklist

Tool Scores

• search_docsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses it does not index and relationship to lookup results. No annotations exist, so description carries full burden. Lacks details on read-only nature, rate limits, or authentication.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with key action. No unnecessary words. Efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for low complexity tool with full schema. Explains purpose and behavior relative to lookup. Lacks output format details, though implied by 'list PDF links'. No output schema to compensate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already fully describes parameters (vendor, query). Description adds little beyond mentioning 'vendor site', which aligns with vendor parameter. Baseline score applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it lists PDF links for a part from vendor site. Distinguishes itself from indexing. However, could more explicitly differentiate from sibling tools like list_indexed_documents and lookup_doc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides context for use after lookup that returns suggestions. But does not explicitly state when not to use or compare to alternatives like lookup_doc or query_doc_content.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_doc_contentA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses the search nature and result fields but does not elaborate on non-obvious behaviors like rate limits, authentication, or destructive potential. The tool is inherently read-only, which is implied but not stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each with a clear purpose: state the tool's function, describe output fields, and specify prerequisites. No redundant information; front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 5 parameters and no output schema, the description covers the main purpose, output structure, and inter-tool dependencies. It lacks details on query syntax or ranking, but the schema descriptions for parameters are thorough. Overall, adequately complete for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema coverage is 100% with descriptions already explaining each param. The description adds context about requiring indexing and output format, which aids understanding but does not significantly enhance parameter meaning beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it performs 'BM25 full-text search over indexed chunks' with specific result fields. It distinguishes itself from siblings by explicitly mentioning the use of read_doc_page for full text and the prerequisite of indexing via read_doc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool (for searching indexed chunks) and what prerequisites exist (PDFs must be indexed via read_doc). It also suggests a follow-up tool (read_doc_page). However, it does not explicitly state when to avoid this tool or compare directly to siblings like search_docs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_doc_pageA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the read-only operation (implied by 'return'), requires indexed documents, and mentions truncation behavior (maxChars cap, truncated flag). It does not detail rate limits or error handling, but for a read retrieval tool, this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no redundancy. Each part serves a purpose: purpose statement, prerequisite, parameter guidance, and usage context. Perfectly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the description covers purpose and guidelines, it lacks information about the output structure (beyond mentioning truncated flag) and error conditions (e.g., invalid page, unindexed document). Given no output schema, more details would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds context beyond the schema: docUrl must be exactly as in search results, pageEnd is optional for a range, maxChars default and hard cap with truncation flag. This aids correct parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states it returns full indexed plain text for a PDF page or page range, and references query_doc_content for getting page numbers. It clearly distinguishes from siblings by noting usage when snippets are too short.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit use case: when snippets are too short for tables or register maps. It also mentions prerequisites (document must be indexed, docUrl must be as in search results). It could be improved by explicitly stating when not to use (e.g., for whole documents, use read_doc).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_docA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description adds behavioral context: downloads and indexes a PDF, requires direct URL, recommends part to avoid UNKNOWN rows. Hints at side effects (indexing) and prerequisites.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise, front-loaded, every sentence adds value. No redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers usage context, parameter emphasis, follow-up actions. Lacks output description, but no output schema is provided. Complete enough for effective agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but description adds value by emphasizing 'part' as strongly recommended and clarifying docIdOrUrl as PDF URL. Enhances understanding beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool downloads and indexes a PDF by direct URL, specifying it's for TI links and PDFs not found via lookup. Distinguishes from sibling tools like lookup_doc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly describes when to use (direct URL required, better than lookup with exact link) and what to do after (use query_doc_content or read_doc_page). Lacks explicit when-not-to-use, but context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_indexed_documentsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description bears full burden. It discloses that the tool performs a local database read (no web fetch) and returns specific fields. For a read-only listing tool, this adequately covers behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (three sentences), front-loaded with the most important information, and every sentence serves a purpose without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description fully explains the return structure (count and documents with specific fields). It covers optional filtering and key traits, making it complete for this simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but description adds value by explaining 'part' is normalized like other tools and omitting it lists all documents for the vendor. This provides context beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool lists indexed PDF metadata from the local database (no web fetch), distinguishes from siblings by emphasizing local-only operation, and clearly specifies return fields (count, documents with id, part, title, docType, url, indexedAt).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use (listing indexed docs locally) and guidance on the optional 'part' filter. It lacks explicit when-not-to-use or alternative tools, but the context is sufficient for basic decisions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_docA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses key behaviors: FTS over local index only, no PDF indexing, returns chunks vs suggestedDocuments, and that maxDocsToIndex is ignored. No annotations provided, so description carries full burden and fulfills it well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise yet comprehensive, using bold for emphasis and a logical flow. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Fully contextualizes the tool for an AI agent: explains behavior, limitations, next steps, and references a resource guide. No output schema or annotations, but description compensates completely.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Adds meaning beyond schema: specifies vendor values, explains part revision normalization, clarifies question usage, and indicates maxDocsToIndex is legacy/ignored. Schema coverage is 100%, but description provides crucial additional context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it performs full-text search over a local index for part numbers and questions, distinguishing it from sibling tools by explicitly noting it does not download or index PDFs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use and when-not-to-use guidance, including next steps (read_doc, query_doc_content) and mentions a known limitation (TI symlink datasheets may be missing from suggestions).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.