Server Quality Checklist

Tool Scores

• calypso-list-knowledge-bucketsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of disclosing behavior. It indicates a read-only listing operation and clarifies scope (team tied to the API key), but does not disclose potential rate limits, pagination, or default behavior nuances beyond the schema. It is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the core action. The bracketed header repeats the tool name, adding slight redundancy, but every other sentence contributes meaningful usage context. It earns a high score for efficiency without unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, the description provides a complete picture of purpose, usage context, and relationship to other tools. It hints at return values by listing retrievable fields, but does not explicitly describe the response format (e.g., array of objects). This is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with the includeArchived parameter already described in the schema. The description does not add parameter-specific meaning beyond the schema, only mentions the types of data returned (bucket ids, slugs, etc.), which are not directly parameter semantics. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('Lists'), resource ('knowledge buckets'), and scope ('for the team tied to the configured Calypso API key'). It also differentiates from sibling tools by explaining its use before uploads and how it complements model discovery.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: 'Use this before uploads when you need bucket ids, slugs, names, member counts, or bucket-store readiness.' It also contrasts with RAG model discovery. However, it lacks explicit when-not-to-use or alternative tool exclusions, so it falls short of a full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calypso-upload-knowledge-files-batchA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses write side-effects (durable store, indexing queue), batch status return, optional polling with terminal states, and bucket field override rules. It does not mention all possible failure semantics or idempotency behavior in depth, but the core workflow is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than minimal but every sentence serves a purpose: purpose, usage, file-source selection, bucket rules, and status behavior. It is well-structured and front-loaded with the main action, then details, with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema or annotations, the description is remarkably complete: it covers the tool's operation, return behavior, polling, and preconditions (bucket destination). It could add more on error handling or what 'batch-level status' contains, but it covers the essentials for a batch upload tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds important contextual meaning: when to use filePath vs contentBase64, and that shared bucket fields are overridden by item-specific fields. This enriches parameter understanding beyond the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool uploads 1 to 100 files into the durable knowledge store and indexing queue in one request, with the explicit use case of bulk corpus ingestion. This distinguishes it from the sibling single-file upload tool by scope and batch behavior.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage guidance: use for bulk corpus ingestion. It also provides specific direction on choosing filePath for local MCP installs and contentBase64 for hosted/remote clients, plus a warning not to base64 local files. However, it does not explicitly reference the singular alternative tool (calypso-upload-knowledge-file), so it lacks full alternative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calypso-rag-agentA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully explains behavior: maintains conversation state, chains follow-up turns via previous_response_id, uses rag_policy for fileIds, and supports /new to reset context. This goes well beyond what the schema reveals.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is fairly long but organized into sections (Responses API, MCP session, quick commands). Every section adds value, though some repetition (model name listed twice) and length could be trimmed slightly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity and no output schema/annotations, the description covers conversation state, file handling, and usage examples. Missing return format details, but the overall context is sufficiently complete for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds meaning by explaining fileIds trigger rag_policy retrieval semantics, and clarifies model default. This enriches the parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool's verb and resource: 'Sends each prompt directly to the Calypso RAG agent using the full conversation context.' It clearly distinguishes this from sibling tools (which list/upload knowledge files) by focusing on retrieval and grounded answers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit usage context: 'Use this when you want Calypso knowledge retrieval and grounded answers from the RAG backend.' It provides typical request examples and quick commands. However, it does not explicitly mention when NOT to use it or name alternative tools for other actions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calypso-upload-knowledge-fileA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the durable bucket-backed nature, the indexing pipeline, return metadata, the requirement to choose exactly one file source, and explains how filePath avoids base64 conversion. It also mentions idempotency for retries. Some aspects like error conditions or permission needs are not covered, but the description adds significant behavioral context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and then efficiently covers usage guidance, parameter selection, and constraints in a compact block. Every sentence adds value; no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite 13 parameters and no output schema, the description is complete: it states what the tool returns, when to use each source parameter, bucket requirements, and optional behaviors like waitForIndexing and createMissingBuckets. It addresses the complexity effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3, but the description adds substantial meaning: it explains the trade-offs between filePath and contentBase64, clarifies the bucket alternatives (bucketIds, bucketSlugs, bucket), and notes the idempotency key's purpose. This goes beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Uploads a file into the durable bucket-backed knowledge store and indexing pipeline', which is a specific verb+resource action. It also distinguishes from siblings by noting this is for the broader knowledge corpus rather than a single RAG chat turn, and indirectly from the batch upload tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is provided: 'Use this when you want a file indexed into the broader knowledge corpus instead of attached directly to a single RAG chat turn.' It also details when to use filePath vs contentBase64 with concrete examples (local vs hosted/remote), and implies batch tool for multiple files. No alternatives are excluded but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.