Server Quality Checklist

Tool Scores

• remove_directory_from_indexC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must carry full burden. States it removes entries but does not disclose if the operation is reversible, impacts other data, or requires permissions. Lacks critical behavioral details for a destructive action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, very concise, but lacks depth. While efficient, it fails to include necessary context that would justify its brevity. Not under-specified but could add more value in the same space.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Output schema exists, so return values are covered, but the description omits behavior like confirmation, status, or side effects. For a destructive tool, this is insufficient. Lacks completeness given the simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and description does not elaborate on the single parameter 'directory_path' beyond its name. No format, examples, or constraints are provided, leaving the agent guessing about valid inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool removes documents and images from the index for a specific directory, which is a specific verb+resource. It distinguishes from siblings like index_directory (adds) and list_indexed_directories (lists).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives, no prerequisites or exclusions. The description only implies usage without clarifying context or providing when-not scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• index_directoryB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description must fully disclose behavioral traits. It mentions that it generates embeddings and stores them, but fails to disclose side effects such as overwriting behavior, performance implications for large directories, or whether it modifies files. Important safety and idempotency details are missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (three lines) and front-loaded with the main action. It includes a structured 'Args' section. No unnecessary words, but could be slightly more compact.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's medium complexity and the presence of an output schema, the description covers the basic functionality well but omits constraints like directory existence, permission requirements, and handling of nested directories. It is adequate but not fully comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning beyond the schema by specifying that directory_path must be an absolute path and providing examples for the ignore parameter (e.g., '*.log', 'drafts'). This compensates for the schema's lack of property descriptions (0% coverage).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool scans a local directory, extracts text from multiple file types (PDF, DOCX, TXT, MD, and raw media bytes), generates semantic embeddings using Gemini 2, and stores them for searching. It uses specific verbs and resources, and the function is distinct from siblings like listing or removing indexes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is provided about when to use this tool versus alternatives. It does not mention prerequisites (e.g., directory existence, permissions) or edge cases like re-indexing. The description only explains what it does, not when it should or should not be used.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_indexed_directoriesB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations present, so the description must carry the burden. It states it lists all indexed file paths, implying a read-only operation, but does not disclose any behavioral traits like authentication needs, rate limits, or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence, concise and front-loaded. No wasted words, though slightly minimal.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and presence of an output schema, the description is minimally adequate. It explains what the tool does, but lacks details like ordering or pagination behavior. Sibling tools provide enough contrast.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so schema coverage is complete. The description adds no extra parameter info but is adequate given the absence of parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'lists' and the resource 'file paths that have been indexed in the database'. It distinguishes itself from siblings like index_directory, remove_directory_from_index, search_my_documents, and sync_indexed_directories.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. No explicit context or exclusions are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_my_documentsB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses use of Gemini 2 embeddings and that it searches both documents and images, but does not mention read-only nature, return format, or pagination. Basic but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence with no wasted words. Immediately communicates core functionality in a front-loaded manner.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Output schema exists (so return format not needed), but description omits parameter explanations and usage context. For a simple search tool, this is adequate but incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% and description does not mention any parameters (query, limit). It adds no meaning beyond the input schema, leaving agents without guidance on what query or limit do.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'semantic search' over 'previously indexed local documents AND images', specifying the verb and resource. It distinguishes from sibling tools (all indexing operations) by being the only search tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description implies usage after indexing (previously indexed) and mentions semantic search, but does not explicitly state when to use (e.g., 'after indexing') or when not to use. No alternatives given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sync_indexed_directoriesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the core behavior: finds unique parent directories and re-indexes them. However, it lacks details on safety (e.g., is it idempotent?), performance, or side effects such as deleting old entries.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action 'Auto-updates existing folders.' Every clause adds meaning with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is mostly complete for a simple sync action with no parameters. However, it could mention that it is typically run after changes to indexed directories, and it does not describe the return value (though an output schema exists).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, and schema description coverage is 100%. The description does not add parameter info (none needed), but baseline for 0 params is 4. No extra semantic value required.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool auto-updates existing folders by re-indexing parent directories of currently indexed files to capture new or modified files. It distinguishes from sibling tools like index_directory (adds a directory) and list_indexed_directories (lists), showing specific verb+resource.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use (to sync after changes to indexed directories), but provides no explicit guidance on when not to use or alternatives. It does not compare to siblings like index_directory or remove_directory_from_index.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.