Server Quality Checklist

Tool Scores

• searchA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes the hybrid search algorithm, output format (ranked results, relevance scores, excerpts, citations), and includes note about pre-formatted citations. No annotations provided, so it carries full burden. Could be improved by explicitly stating it is read-only and has no side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with front-loaded core function. Every word adds value; no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a search tool with filters and an output schema, the description adequately covers purpose and output but could mention default behavior of searching all collections when no collection specified (though schema covers it). Overall sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the description need not add much. It does not elaborate on parameters beyond what the schema already provides, giving no extra meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly specifies the action (Search), the resource (Point's verified knowledge base), and the method (hybrid search). It distinguishes itself from sibling tools like get_document_full which focus on retrieving documents rather than searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus sibling tools. The description does not provide usage context or alternatives, leaving the agent to infer usage from tool names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_document_tocA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses the lightweight nature (~50 tokens) and that it returns chunk IDs. While it could detail permissions or rate limits, the behavioral impact is minimal for a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with the action, and every sentence adds unique value (purpose, usage hint, output description). No fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (noted in context signals), the description covers the key aspects: what the tool does, when to use it, what it returns, and how it relates to siblings. Fully sufficient for effective tool selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline 3. The description does not add info beyond the schema's existing description for 'doc_id', which already explains the source.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'Get a lightweight table of contents' which is specific and actionable. Distinguishes from siblings like get_document_full (full document) and get_sections (specific sections) by noting it returns chunk IDs for use with get_sections.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to 'Use this to understand document structure before loading specific sections,' providing clear context. Implicitly suggests not to use when full content is needed, and links to get_sections as a follow-up.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_document_fullA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description bears full burden. It discloses that content can be large (thousands of tokens) and advises caution. However, it could explicitly state it's a read-only operation, though implied by 'get'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: one for purpose, one for warning. No fluff, front-loaded with key information. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given output schema exists (context indicates true), description needn't cover return values. The description fully addresses usage, warnings, and purpose, making it complete for a retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 1 parameter (doc_id) with description, and schema coverage is 100%. The description adds no extra parameter details beyond the schema, but baseline 3 applies as schema handles it well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it gets the full markdown content of a document, with a verb and resource. It distinguishes from siblings by naming alternatives (get_document_toc + get_sections) and warns about size, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use (complete text needed) and when not (prefer targeted retrieval via TOC+sections). Includes a warning about large size, guiding the agent to consider alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_collectionsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, but the description fully discloses the two behavioral modes (list all vs. semantic search) and implies read-only operation. It does not cover rate limits or auth, but for a simple browsing tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short paragraphs with no wasted words. The front-loaded summary ('Browse or search available knowledge collections') immediately conveys the overall purpose, and the mode details are efficiently presented.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (1 optional parameter, output schema present), the description fully covers the tool's functionality. It explains both modes clearly, and the output schema handles return value details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage and describes the 'query' parameter. The description adds value beyond the schema by explaining how the parameter's presence or absence changes the tool's behavior (list all vs. search).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool browses or searches collections, and differentiates between listing all collections (without query) and semantic search (with query). This distinguishes it from sibling tools like 'search' which does full-text document search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use each mode (with or without query). While it does not mention explicit when-not-to-use scenarios, the context of sibling tools implies appropriate usage. A slight improvement would be to contrast with 'search' for document-level queries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_sectionsA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Discloses max 50 per request limit and what each section includes. Could mention read-only nature or error behavior, but sufficient for a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no waste. Front-loaded with purpose, then usage context, then return content. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With output schema present, description need not detail return values. It covers usage context, limit, and content. Complete for a single-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with description of chunk_ids as 'List of chunk IDs to load (get these from search results or TOC). Max 50 per request.' Description adds context beyond schema: source of IDs and limit, enhancing meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'Load specific document sections by their chunk IDs', a specific verb and resource. Differentiates from siblings by specifying use after search or get_document_toc, and implies it is for specific passages not full document or TOC.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: 'Use after search or get_document_toc'. Implies not to use for full document (get_document_full) or TOC (get_document_toc), providing clear context and alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.