Server Quality Checklist

Tool Scores

• searchA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the output format and a requirement, but does not explicitly state that the tool is read-only, nor does it disclose any potential side effects, rate limits, or costs. The description is adequate but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: two sentences covering purpose, use cases, sibling tool distinction, return format, and a prerequisite. No wasted words, front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only 2 parameters and an output schema (reducing need to explain returns), the description covers purpose, usage, and a constraint. However, it lacks explanation for the limit parameter and does not elaborate on the output schema or error conditions. Still, it is fairly complete for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must clarify parameters. Only the query parameter is implied by the semantic search context; the limit parameter is not explained at all (default 5, but no indication of its effect on results).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is for semantic search across indexed files, contrasting with search_keyword. It specifies the return format (ranked chunks with file path, file_id, cosine distance).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises use for conceptual questions where exact wording is unknown, and directs to use search_keyword for exact terms. Also notes a prerequisite (ollama server must be running).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_filesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It states behavior: returns file_id, size, path, largest first; explains name_contains matches anywhere in full path. Lacks auth, rate limits, but for a read-only listing, these are less critical.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is one paragraph but well-structured: purpose, filter explanation, use cases, return info. Could be slightly more concise, but every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool is simple (3 params, output schema exists), the description covers the key aspects: purpose, filters, return fields, ordering. It is complete enough for an agent to understand and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must explain parameters. It explains extension (no leading dot) and name_contains (substring anywhere in path). The limit parameter is not explained, but its default and type are in schema. Description adds value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it's for browsing indexed files by metadata, not content search. It distinguishes from content search and gives specific filter examples (extension, name_contains). Verb 'Browse' and resource 'indexed files' are specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description says 'Use for exploration like...' and explicitly states 'not a content search', implying when to use this tool vs sibling tools like search. However, it does not explicitly name alternatives or say when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_keywordA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes supported FTS5 syntax, return format (files with snippet and file_id, ordered by relevance). No annotations provided, so description carries full burden; it is sufficiently transparent for a read-only search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise and well-structured: one sentence for purpose, one for usage guidance, one for syntax, and one for return format. Every sentence adds value without unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 parameters, output schema exists), the description covers purpose, usage, supported syntax, and return format. It distinguishes from siblings and is complete for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but description adds context that keyword supports FTS5 syntax. However, it does not explain the 'limit' parameter or provide details on parameter semantics beyond the overall purpose.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is for exact keyword/phrase search over extracted file text, specifies the use of FTS5 and BM25 ranking, and distinguishes it from the sibling tool 'search' which is for semantic search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use (looking for specific term, function name, literal phrase) and when not to (meaning-based questions, recommending 'search' instead). Provides clear context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_file_contentA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility. It discloses that large files are capped and explains the behavior of the two modes in detail. It does not mention auth requirements or error handling, but provides enough behavioral context for typical usage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, using two short paragraphs and bullet points for clarity. Every sentence provides essential information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists (so return values need not be detailed), the description covers purpose, all parameters, usage modes, and a behavioral caveat about capping. It is complete for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must add meaning for all parameters. It thoroughly explains 'around_chunk' (negative vs. non-negative behavior), 'radius' (number of surrounding chunks), and 'file_id' (from search result). This adds significant value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it reads an indexed file's text by file_id from a search result. It uses specific verbs ('read') and resource ('file's text'), and distinguishes from sibling tools that search or list files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains two modes (whole text vs. chunk expansion) and when to use each, including a recommendation to prefer window mode for large files. It also mentions passing the chunk index from search results. However, it does not explicitly state when not to use this tool or provide direct comparisons with siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.