Server Quality Checklist

Tool Scores

• browse_csdk_docsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It transparently states what the tool returns (list or file content) and how the optional file parameter changes that behavior. It implies a read-only operation without explicitly claiming side effects, which is sufficient for a documentation browsing tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loads the purpose and return behavior, and includes a usage note. Every sentence earns its place with no redundancy or vague filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (2 parameters, no output schema), and the description covers the core purpose, return modes, and usage context. The sibling tools provide contrast, and the description explains enough for an agent to select and invoke it correctly. Minor gaps (e.g., error handling) are not critical for this use case.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with detailed descriptions for both parameters, including the example values and the 'omit to list all files' behavior. The description adds no new parameter information beyond what the schema already provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Browse CSDK documentation by category') and specifies the two possible return types: a list of files or full content of a specific file. It also distinguishes from the sibling tool search_csdk_docs by positioning it as a direct browsing alternative, satisfying the goal of differentiating tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear usage trigger: 'Use this when you know which category or file you need, instead of searching.' This explicitly tells the agent when to use this tool and when to prefer search, providing practical guidance beyond what the schema offers.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_csdk_docsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the TF-IDF search mechanism and that it returns documentation chunks, which is transparent about core behavior. However, it does not explicitly state side effects or safety (e.g., read-only), though this is implied for a search operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action, and each sentence adds value. The phrase 'Search ... using TF-IDF search' has slight redundancy, but it is minor and does not detract from overall conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description adequately covers purpose, usage, and return type ('documentation chunks'). There is no output schema, but the description gives a reasonable expectation of what the tool returns. For a search tool with straightforward parameters, this is sufficient, though it could mention result structure in more detail.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description does not add parameter-specific meaning beyond what the schema already provides, such as query examples or filtering options, but the schema descriptions are thorough enough.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches Sisense Compose SDK documentation using TF-IDF search and returns relevant chunks. It uses a specific verb ('search') and resource ('documentation'), and distinguishes itself from sibling tools like browse_csdk_docs and list_csdk_topics by focusing on targeted search rather than browsing or listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides usage context: 'Use this for specific questions where you need targeted results.' This implies the tool is ideal for focused queries but does not explicitly name alternatives or exclusions, making it clear but not fully comprehensive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_csdk_topicsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It accurately describes the operation as listing categories and files, which implies a read-only action with no side effects. While it does not detail output format or limitations, for a simple list tool this is adequate and concise.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action('List all available...') followed by a usage note. Every word serves a purpose and there is no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema, no annotations), the description adequately explains what it does and why to use it. It is complete enough for an agent to select and invoke it correctly in the context of browsing or searching documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so there is nothing to describe. Per the rubric, the baseline for no-parameter tools is 4, and the description adds context about understanding the documentation structure, which is helpful.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists all available documentation categories and files, and explicitly positions it as a precursor to browsing or searching, distinguishing it from sibling tools search_csdk_docs and browse_csdk_docs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says 'Use this to understand the documentation structure before browsing or searching,' which provides explicit when-to-use guidance and implies the alternatives (browse and search tools). This gives clear context for selecting this tool first.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.