Server Quality Checklist

Tool Scores

• ragbrain_list_namespacesA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must cover behavioral aspects. It mentions return fields but does not disclose whether the operation is read-only, has side effects, or requires permissions. The optional parameter 'tree_view' is not mentioned in the description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences, front-loads the purpose, and contains no extraneous information. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (listing namespaces with one optional param), the description covers the main action and return types. However, it does not explain the hierarchy structure or the effect of the 'tree_view' parameter, and it does not differentiate from sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with the parameter 'tree_view' already described. The description adds no additional meaning beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists all namespaces and specifies the returned fields (names, descriptions, counts, hierarchy). It also indicates usage for discovering knowledge areas. However, it does not explicitly differentiate from sibling tools like browse_namespace.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says 'Use this to discover what knowledge areas are available,' which gives a usage context. But it lacks explicit guidance on when not to use this tool or what alternatives exist (e.g., browsing a specific namespace).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ragbrain_searchA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavioral traits. It clarifies the tool performs semantic search and returns text chunks, but omits details like read-only nature, authentication requirements, or output format. This is adequate for a simple search tool but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise at two sentences, with no wasted words. It is front-loaded with the core action and result, making it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description could better explain return structure (e.g., chunk content, metadata). It covers the basics but lacks details on pagination or result formatting, making it sufficient but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description does not add new meaning to parameters beyond the schema; it repeats 'query' and implies results but offers no additional semantic value for parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs semantic search across the RAGBrain knowledge base, returning relevant text chunks. This distinguishes it from sibling tools like ragbrain_browse_namespace (navigation) or ragbrain_get_document (retrieval by ID), making its purpose specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says 'Use this to find information on any topic stored in RAGBrain,' providing a clear general context. However, it does not explicitly mention when not to use this tool (e.g., for precise document retrieval via ragbrain_get_document) or suggest alternatives, leaving room for ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ragbrain_browse_namespaceA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description is the sole source for behavioral info. It discloses that the tool is a read operation (listing documents) and mentions return fields. However, it does not clarify pagination behavior (though the limit parameter is documented in schema) or whether there are any other side effects or authentication needs. Given no annotations, a score of 3 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise: two sentences. The first sentence states the core function and return values; the second provides usage guidance. No unnecessary words. It is front-loaded with the main purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema, so the description must explain return values. It does so by listing names, IDs, chunk counts, and creation dates. However, it does not mention ordering or whether there is pagination beyond the limit parameter. For a browsing tool, this is a minor gap. Score 3 means adequate but could be more complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (both 'namespace' and 'limit' have descriptions). The description adds context by explaining namespace as 'a particular knowledge area' with examples ('personal', 'work/projects'), which adds meaning beyond the schema. The limit parameter details (default 50, max 200) are in schema, but the description reinforces usage. Slight room for improvement if description clarified the effect of limit on results.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: listing documents in a namespace. It specifies the verb 'list' and the resource 'documents in a namespace', and mentions return fields (names, IDs, chunk counts, dates). It distinguishes from siblings like ragbrain_search (query-based) and ragbrain_list_namespaces (lists namespaces, not documents).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a usage hint: 'Use this to see what's in a particular knowledge area.' This implies when to use it but does not mention when to avoid it or explicitly compare to siblings like ragbrain_discover_documents or ragbrain_search. More explicit guidance on alternatives would improve this.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ragbrain_discover_documentsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses the return values (titles, headings, relevance scores) and mentions 'semantic search over summaries'. It does not discuss potential side effects, rate limits, or authentication, but the inferred read-only nature is reasonable. The description adds some behavioral context beyond the schema but could be more thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (under 50 words) and well-structured. The key purpose is front-loaded, followed by usage guidance, example queries, and a clear link to the next step (ragbrain_search). Every sentence serves a purpose with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of output schema, the description adequately explains what is returned (titles, headings, relevance scores). The parameter schema is fully documented. The description also provides example queries and workflow guidance. It could mention pagination or result limits (though top_k is in schema), but overall it is complete enough for an AI agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by providing example queries and explaining the workflow (e.g., using the tool before searching). It also clarifies that the tool returns titles, headings, and scores, which adds meaning beyond the schema definitions. This elevates the score above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Discover documents by semantic search over their summaries.' It uses a specific verb ('Discover') and resource ('documents'), and distinguishes it from sibling tools by specifying it should be used before searching for specific content and directing to 'ragbrain_search' afterward.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use this tool: 'Use this to find documents about a topic BEFORE searching for specific content.' It also gives example queries and names the follow-up sibling tool 'ragbrain_search'. However, it does not explicitly state when not to use it, missing full exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ragbrain_get_documentA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must cover behavioral traits. It states that it 'Returns the complete reconstructed text from all chunks,' which gives a good understanding of the output. Minor missing details like performance implications but adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences with no wasted words. The purpose is front-loaded, and every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description covers purpose, return value (reconstructed text), and usage hint. Could mention handling of missing IDs or size limits, but not essential.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a single parameter (doc_id) already described in the schema as 'The document ID (UUID) to retrieve.' The description adds no further semantic meaning beyond the schema, so baseline applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Retrieve the full content of a specific document by its ID' and distinguishes itself from sibling tools (browse, discover, list, search) by specifying its use for reading an entire document.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this when you need to read an entire document,' providing clear context for when to use it. However, it does not mention when not to use it or suggest alternative tools for partial reads.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.